RAG System Builder

• Outcome: Review
• Latest audit: Sep 11, 2026 · 5:15 AM UTC
• Version: 1.0.0

Overview

The skill is a coherent local RAG guide, but users should review it because its offline claims are overstated and its examples can download mutable model artifacts, persist document text, and expose an unsafe debug web server.

Install only if you are comfortable reviewing and adapting the generated RAG code. Pin Python dependencies and Hugging Face model revisions, download models through a trusted path, remove the runtime remote fallback for true offline use, store indexes in a protected directory, avoid sensitive documents unless retention is acceptable, and do not expose the Flask example with debug mode enabled.

A.I.G

Vulnerability Patterns

• Insecure Dependencies: Introduces malicious components through unsafe dependency sources
• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review

Findings (3)

T08 · Insecure Dependencies

Warning

• Location: SKILL.md:26
• Finding: Unpinned Python Dependencies and Mutable Model Artifacts## Vulnerability Details **File Location**: `SKILL.md:26`, `SKILL.md:46`, `SKILL.md:102`, `SKILL.md:343-346`, `README.md:20-26`, `USAGE.md:20`, and `USAGE.md:234` **Vulnerability Type**: Supply-chain exposure through unpinned dependencies and model revisions **Risk Level**: Medium ### Vulnerable Code `SKILL.md:26` and `SKILL.md:343`: ```bash pip install sentence-transformers faiss-cpu click flask ``` `README.md:20`: ```bash pip install sentence-transformers faiss-cpu click flask ``` `SKILL.md:46`, `README.md:26`, and `USAGE.md:20`: ```bash python -c "from huggingface_hub import snapshot_download; snapshot_download(repo_id='sentence-transformers/all-MiniLM-L6-v2', local_dir='./models/all-MiniLM-L6-v2')" ``` `SKILL.md:102`: ```python self.model = SentenceTransformer(self.model_name) ``` `USAGE.md:234`: ```bash python -c "from huggingface_hub import snapshot_download; snapshot_download(repo_id='sentence-transformers/all-mpnet-base-v2', local_dir='./models/all-mpnet-base-v2')" ``` ### Technical Analysis The installation commands do not pin exact package versions or verify package hashes. Consequently, running the same documented command at different times can install different dependency versions. The model download commands similarly reference mutable Hugging Face repositories without specifying immutable commit revisions or expected artifact checksums. The `SentenceTransformer(self.model_name)` fallback can also retrieve model artifacts implicitly when the expected local model is unavailable. This weakens the documented offline trust boundary and makes runtime behavior depend on externally supplied content. This finding does not establish that the named packages or models are malicious. The vulnerability is the absence of controls that ensure users receive the same reviewed artifacts. ### Attack Path 1. An attacker compromises an upstream package release, model repository, maintainer acco ...[truncated 1169 chars]
• Remediation: ## Remediation Suggestions 1. Pin every Python dependency to an explicitly reviewed version in a lock file. 2. Generate and enforce cryptographic hashes, such as through `pip install --require-hashes -r requirements.txt`. 3. Use a trusted package index explicitly and apply dependency vulnerability scanning in CI. 4. Pin each Hugging Face download to an immutable commit using the `revision` parameter. 5. Record and verify checksums for all downloaded model files before loading them. 6. Configure runtime loading for local-only operation after verification, for example by using supported `local_files_only` controls. 7. Fail closed when the verified local model is missing instead of silently downloading a replacement. 8. Document the initial network-dependent setup separately from offline runtime behavior.

T09 · Insecure Skill Coding Practices

Warning

• Location: USAGE.md:74
• Finding: Shell Command Injection in Batch-Processing Example## Vulnerability Details **File Location**: `USAGE.md:74-77` **Vulnerability Type**: Shell command injection through unsafe string interpolation **Risk Level**: Medium ### Vulnerable Code ```python # Process multiple folders folders = ["./docs1", "./docs2", "./docs3"] for folder in folders: os.system(f"python rag.py ingest --docs-path {folder}") ``` ### Technical Analysis The example interpolates a folder path directly into a command string passed to `os.system`. This API invokes a system shell, so shell metacharacters in `folder` are interpreted as command syntax rather than as part of one path argument. The example currently uses hardcoded folder names, so those exact values are not exploitable. However, it is presented as a reusable batch-processing pattern. If an implementation obtains folder values from command-line input, configuration, uploaded metadata, filenames, or another untrusted source, an attacker can append shell operators and arbitrary commands. Quoting the value manually would remain fragile across shells and platforms. The correct defense is to avoid shell interpretation entirely. ### Attack Path 1. A developer adopts the documented batch-processing example. 2. The `folders` collection is changed to include values from an untrusted source. 3. An attacker supplies a folder value containing shell metacharacters followed by a command. 4. The f-string constructs a single shell command containing both the intended ingestion command and the attacker-controlled command. 5. `os.system` passes the string to the operating-system shell. 6. The shell executes the injected command with the privileges of the RAG process. ### Impact Assessment Successful exploitation provides arbitrary command execution under the account running the batch-processing script. An attacker could read, modify, or delete accessible files; tamper with the RAG project or vector store; access documents available to the proce ...[truncated 230 chars]
• Remediation: ## Remediation Suggestions Replace shell-based execution with an argument array: ```python import subprocess import sys folders = ["./docs1", "./docs2", "./docs3"] for folder in folders: subprocess.run( [sys.executable, "rag.py", "ingest", "--docs-path", folder], check=True, ) ``` Additionally: 1. Validate that each input resolves to an allowed directory. 2. Apply an explicit base-directory restriction when folders may be user-controlled. 3. Reject nonexistent paths and unexpected path types. 4. Do not enable `shell=True` in the replacement implementation. 5. Handle `subprocess.CalledProcessError` explicitly and log failures without exposing sensitive path information.

T09 · Insecure Skill Coding Practices

Note

• Location: USAGE.md:106
• Finding: Flask Development Debugger Enabled in Web Interface Template## Vulnerability Details **File Location**: `USAGE.md:106-107` **Vulnerability Type**: Unsafe debug configuration **Risk Level**: Low ### Vulnerable Code ```python if __name__ == '__main__': app.run(debug=True, port=5000) ``` ### Technical Analysis The documented Flask application starts with development debug mode enabled. Debug mode can expose detailed exception pages containing stack traces, source excerpts, filesystem paths, and application state. Flask's development server and debugger are not designed for production deployment. The shown call uses Flask's default loopback binding, which reduces immediate remote exposure. Risk arises when users adapt the example by changing the bind address, placing it behind a proxy, running it in a shared environment, or otherwise making the service reachable while retaining `debug=True`. ### Attack Path 1. A user deploys or exposes the supplied web-interface template without disabling debug mode. 2. An attacker sends malformed requests or inputs that trigger an unhandled exception. 3. Flask returns a detailed debugger response. 4. The attacker extracts sensitive diagnostic information from the response. 5. If interactive debugger functionality is reachable and its protection is bypassed or misconfigured, the attacker may be able to execute Python code in the server process. ### Impact Assessment The direct impact includes disclosure of source code fragments, local paths, internal implementation details, and values present in exception context. In a remotely exposed and inadequately protected deployment, debugger abuse can potentially lead to arbitrary code execution with the privileges of the Flask service account. In the exact default loopback-only example, exploitation generally requires local access or an additional exposure mechanism. This constraint lowers the finding's severity but does not make debug mode suitable for deployment.
• Remediation: ## Remediation Suggestions 1. Disable debug mode by default: ```python if __name__ == "__main__": app.run(debug=False, port=5000) ``` 2. Keep development and production configuration separate. 3. Deploy through a production WSGI server rather than Flask's development server. 4. Apply authentication and authorization before exposing document-query endpoints. 5. Restrict network binding and firewall access to trusted clients. 6. Add centralized exception handling that returns generic errors while recording protected server-side logs. 7. Validate `query` and `top_k` request fields and impose request-size and rate limits before deployment.

SkillSpector

Vulnerability Patterns

• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• MCP Tool Poisoning: Hidden Instructions, Unicode Deception, Parameter Description Injection
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access
• Supply Chain: Unpinned Dependencies, External Script Fetching, Obfuscated Code

Findings (8)

Description-Behavior Mismatch

Medium

• Confidence: 95% confidence
• Finding: The README makes a strong 'complete offline' claim while the documented setup instructs users to download a model from Hugging Face, which requires network access and introduces a supply-chain dependency. This discrepancy can mislead users operating in air-gapped or security-sensitive environments into trusting the setup as offline-safe when it is not.

Description-Behavior Mismatch

Medium

• Confidence: 96% confidence
• Finding: The skill explicitly advertises that it works 'completely offline,' but the provided embedding loader falls back to loading from Hugging Face when the local model is missing. In offline, privacy-sensitive, or regulated environments, this can silently trigger unexpected outbound network access and break security assumptions about isolation and data handling.

Missing User Warnings

Medium

• Confidence: 94% confidence
• Finding: The skill presents itself as offline, yet the documented model download step contacts Hugging Face and transmits network metadata such as IP address and request details. Without a clear warning, users may run this in environments where any external contact is prohibited or where privacy expectations require fully air-gapped behavior.

Missing User Warnings

Medium

• Confidence: 95% confidence
• Finding: The skill ingests documents and stores metadata plus chunk text to local disk, but the description does not clearly warn users that potentially sensitive document contents will be persisted. In a RAG context, this can expose confidential data through local files, backups, shared workstations, or improper filesystem permissions.

Description-Behavior Mismatch

Medium

• Confidence: 94% confidence
• Finding: The manifest describes building and deploying local RAG systems with offline document processing, but the usage instructions explicitly rely on `huggingface_hub.snapshot_download` to fetch models from a remote service. While model acquisition can be a setup step, presenting network-dependent downloading as core usage conflicts with the manifest's offline emphasis.

Missing User Warnings

Medium

• Confidence: 98% confidence
• Finding: Running Flask with debug=True exposes the Werkzeug debugger, which can lead to sensitive information disclosure and, in some deployment scenarios, remote code execution if the service is reachable by untrusted users. In a skill intended to help users deploy a query interface, omitting any warning makes it more likely that users will run this unsafe configuration outside of a local-only development environment.

Intent-Code Divergence

Low

• Confidence: 88% confidence
• Finding: The docstring describes 'local fallback,' but the implementation actually performs a remote Hugging Face load if the local path is unavailable. This mismatch is dangerous because operators may trust the code comments and deploy it in environments where any unexpected egress is a policy violation.

Missing User Warnings

Low

• Confidence: 77% confidence
• Finding: The usage guide instructs users to download models from Hugging Face via `snapshot_download`, which initiates an external network call. The document does not disclose that this step contacts a third-party service and transfers request metadata, which is a relevant privacy and network-behavior warning for markdown guidance.

Static analysis

No suspicious patterns detected.