Back to skill

Security audit

rag

Security checks for vulnerabilities and agentic risk

Overview

The skill is a coherent local RAG tool, but its optional LLM mode can send private document excerpts and an API key to any environment-configured endpoint, and it includes under-scoped local usage logging.

Install only if you are comfortable with a RAG skill that indexes the document directory you provide and stores full text chunks in the output index. Keep it in offline mode for sensitive material. If enabling LLM mode, set OPENAI_BASE_URL only to a trusted HTTPS provider you control, use a limited API key, and assume the question, document names, and retrieved excerpts may be sent to that service. Avoid putting sensitive details into learner notes or preferences unless local retention is acceptable.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/rag_synthesize.py:67
Finding

Unvalidated LLM Endpoint Can Receive API Credentials and Private RAG Content

Content
View full analysis

Vulnerability Details

File Location: scripts/rag_synthesize.py, lines 67–90
Vulnerability Type: Arbitrary outbound endpoint and sensitive-data disclosure
Risk Level: Medium

Vulnerable Code

python
key = os.environ.get("OPENAI_API_KEY")
base = os.environ.get("OPENAI_BASE_URL")
model = os.environ.get("OPENAI_MODEL", "gpt-4o-mini")
if not (key and base):
    return None
evidence = "\n\n".join(f"[{i+1}] ({c['doc']})\n{t['text']}" for i, (s, c) in enumerate(hits))
prompt = (
    "你是严谨的检索增强问答助手。只依据下面的【证据】回答,"
    "并在句末用 [n] 标注引用编号。若证据不足,明确说不知道。\n\n"
    f"【证据】\n{evidence}\n\n【问题】\n{question}\n\n【回答】"
)
payload = {"model": model, "messages": [{"role": "user", "content": prompt}], "temperature": 0.2}
req = urllib.request.Request(
    base.rstrip("/") + "/chat/completions",
    data=json.dumps(payload).encode("utf-8"),
    headers={"Authorization": f"Bearer {key}", "Content-Type": "application/json"},
)
try:
    with urllib.request.urlopen(req, timeout=30) as resp:
        data = json.loads(resp.read().decode("utf-8"))
    return data["choices"][0]["message"]["content"].strip()
except Exception as e:
    print(f"⚠️ LLM 调用失败,回退抽取式:{e}")
    return None

Technical Analysis

The value of OPENAI_BASE_URL is used directly to construct the outbound API URL without validating its scheme, hostname, resolved address, or trust level. The request includes the OPENAI_API_KEY as a bearer token. Its body contains the user's question, retrieved document names, and retrieved document contents.

An attacker who can influence this environment variable can redirect the request to an attacker-controlled server. The code also accepts plain HTTP URLs, allowing the authorization token and RAG data to be exposed through network interception. A URL targeting an internal service could additionally cause the application to issue an unintended server-side request, although the fixed /chat/completions suffix and JSON request format limit the reachable interfac ...[truncated 1468 chars]

Remediation
View remediation

Remediation Suggestions

  1. Parse the configured URL with urllib.parse.urlsplit and permit only https.
  2. Reject malformed URLs, embedded user information, fragments, unexpected ports, and unsupported schemes.
  3. Maintain an explicit allowlist of trusted LLM API hostnames rather than accepting arbitrary destinations.
  4. Resolve the hostname and reject loopback, link-local, multicast, unspecified, and private-network addresses unless an administrator has explicitly enabled a documented private endpoint.
  5. Revalidate redirects or disable them so a trusted URL cannot redirect requests to an untrusted host.
  6. Display the destination and explain that the question and retrieved document excerpts will leave the machine before the first external request; require explicit user consent.
  7. Apply data minimization by limiting excerpt size and allowing users to exclude sensitive documents from external generation.
  8. Avoid sending the API key over plain HTTP under all circumstances.
  9. Document precisely which fields are transmitted and provide a configuration option that forcibly disables network generation.
  10. Use provider credentials with minimal permissions, quotas, rotation procedures, and monitoring to constrain the impact of disclosure.
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Taint TrackingDirect Taint Flow, Variable-Mediated Taint Flow, Credential Exfiltration Chain
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Findings (14)

Tainted flow: 'req' from os.environ.get (line 75, credential/environment) → urllib.request.urlopen (network output)

Critical
Category
Data Flow
Confidence
96% confidence
Finding

The code constructs the destination URL from OPENAI_BASE_URL and sends both the bearer token and retrieved document content to that endpoint via urllib.request.urlopen. Because the endpoint is fully environment-controlled and not validated, a misconfigured or attacker-influenced environment can exfiltrate credentials and sensitive local document contents to an arbitrary server; in a RAG skill, this context makes the issue more dangerous because retrieved chunks may contain proprietary or confidential material.

Content

Scanner excerpt · scripts/rag_synthesize.py (reported line 81)May include surrounding context.

python
headers={"Authorization": f"Bearer {key}", "Content-Type": "application/json"},
    )
    try:
        with urllib.request.urlopen(req, timeout=30) as resp:
            data = json.loads(resp.read().decode("utf-8"))
        return data["choices"][0]["message"]["content"].strip()
    except Exception as e:

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The documented behavior claims a focused RAG workflow, but the skill also describes a built-in learner that records operation details after each retrieval/synthesis. Undeclared logging or preference/usage recording is security-relevant because it can persist sensitive queries, document-derived content, or user workflow metadata beyond the stated purpose of answering from local documents.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
93% confidence
Finding

The documented behavior claims a focused RAG workflow, but the skill also describes a built-in learner that records operation details after each retrieval/synthesis. Undeclared logging or preference/usage recording is security-relevant because it can persist sensitive queries, document-derived content, or user workflow metadata beyond the stated purpose of answering from local documents.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

This file implements a generic cross-skill telemetry and self-learning component, not functionality required for RAG indexing, retrieval, or source-cited answering. In the context of a RAG skill, this scope mismatch is dangerous because it introduces persistent behavior-tracking capabilities that expand data collection and attack surface beyond the user-declared purpose, undermining least privilege and informed consent.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
87% confidence
Finding

The skill advertises capabilities that involve reading local files, writing indexes/answers, accessing environment variables, and optionally making network requests, but it declares no explicit tool scope or permissions. That creates an authorization/expectation gap: an agent or user may invoke a publicly visible skill without clear boundaries on what resources it can access, increasing the chance of overbroad file access or unintended external calls.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The skill explicitly supports an optional LLM mode that injects retrieved document context into prompts sent to an external service, but the markdown does not warn users that local document excerpts may leave the host. In a RAG context, retrieved chunks often contain proprietary, regulated, or personal data, so omission of this disclosure can lead to unintended exfiltration of sensitive material.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The docstring explicitly states the module is skill-agnostic and usable by any WorkBuddy skill, which conflicts with its packaging inside a RAG skill. That discrepancy is a security concern because it indicates hidden shared infrastructure and broad reuse potential, increasing the risk that a user enables a narrow RAG capability while unknowingly installing a general-purpose tracking component.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

This Python file contains natural-language descriptions and output strings entirely in Chinese, including the module description and command help text. Because the skill does not provide any user opt-in, alternative locale, or justification that it is intended only for Chinese-speaking users, it conflicts with the policy against forcing a specific language without user choice.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The code persistently stores usage history, error notes, and user preferences in learned_patterns.json, even though such telemetry is not justified by the stated RAG purpose. If notes or preferences contain sensitive prompts, file paths, document snippets, or operational details, this creates unnecessary local data retention and possible cross-context privacy leakage.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

When LLM mode is enabled, the script transmits the user's question plus retrieved document text to an external API without an explicit runtime notice or consent step. In this skill, the retrieved context comes from local indexed documents, so the transmitted evidence may include sensitive internal data, making silent exfiltration to third-party services a real privacy and security risk.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The prompt hardcodes Chinese instructions and answer framing for the assistant, which imposes a specific language/locale behavior. There is no opt-in, language selection mechanism, or documented reason that the skill must operate only in Chinese.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The docstring at L68 says the function returns [(start, end, text)], implying positional metadata is preserved for each chunk. However, the implementation appends only "".join(piece) at L79, so callers receive plain strings with no start/end offsets, which directly contradicts the documented behavior.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
95% confidence
Finding

This code file contains user-facing natural-language documentation and command help entirely in Chinese, including the usage description and argument help strings. Under the stated policy, forcing a specific language without user opt-in is a natural-language policy violation unless the locale constraint is explicitly justified.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
93% confidence
Finding

This code file contains natural-language documentation and status messages entirely in Chinese, including the module docstring and printed user feedback. Under the policy rule, forcing a specific language without opt-in is a natural-language policy violation because the skill does not offer any locale or language selection.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.