RAG System Builder

• Outcome: Review
• Latest audit: Sep 11, 2026 · 5:15 AM UTC
• Version: 1.0.0

Overview

The skill is a coherent local RAG guide, but users should review it because its offline claims are overstated and its examples can download mutable model artifacts, persist document text, and expose an unsafe debug web server.

Install only if you are comfortable reviewing and adapting the generated RAG code. Pin Python dependencies and Hugging Face model revisions, download models through a trusted path, remove the runtime remote fallback for true offline use, store indexes in a protected directory, avoid sensitive documents unless retention is acceptable, and do not expose the Flask example with debug mode enabled.

A.I.G

Vulnerability Patterns

• Insecure Dependencies: Introduces malicious components through unsafe dependency sources
• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review

Findings (3)

T08 · Insecure Dependencies

Warning

• Location: SKILL.md:26

Finding

Unpinned Python Dependencies and Mutable Model Artifacts

Content

View full analysis

Vulnerability Details

File Location: SKILL.md:26, SKILL.md:46, SKILL.md:102, SKILL.md:343-346, README.md:20-26, USAGE.md:20, and USAGE.md:234
Vulnerability Type: Supply-chain exposure through unpinned dependencies and model revisions
Risk Level: Medium

Vulnerable Code

SKILL.md:26 and SKILL.md:343:

bash

pip install sentence-transformers faiss-cpu click flask

README.md:20:

bash

pip install sentence-transformers faiss-cpu click flask

SKILL.md:46, README.md:26, and USAGE.md:20:

bash

python -c "from huggingface_hub import snapshot_download; snapshot_download(repo_id='sentence-transformers/all-MiniLM-L6-v2', local_dir='./models/all-MiniLM-L6-v2')"

SKILL.md:102:

python

self.model = SentenceTransformer(self.model_name)

USAGE.md:234:

bash

python -c "from huggingface_hub import snapshot_download; snapshot_download(repo_id='sentence-transformers/all-mpnet-base-v2', local_dir='./models/all-mpnet-base-v2')"

Technical Analysis

The installation commands do not pin exact package versions or verify package hashes. Consequently, running the same documented command at different times can install different dependency versions. The model download commands similarly reference mutable Hugging Face repositories without specifying immutable commit revisions or expected artifact checksums.

The SentenceTransformer(self.model_name) fallback can also retrieve model artifacts implicitly when the expected local model is unavailable. This weakens the documented offline trust boundary and makes runtime behavior depend on externally supplied content.

This finding does not establish that the named packages or models are malicious. The vulnerability is the absence of controls that ensure users receive the same reviewed artifacts.

Attack Path

1. An attacker compromises an upstream package release, model repository, maintainer acco ...[truncated 1169 chars]

Remediation

View remediation

Remediation Suggestions

1. Pin every Python dependency to an explicitly reviewed version in a lock file.
2. Generate and enforce cryptographic hashes, such as through pip install --require-hashes -r requirements.txt.
3. Use a trusted package index explicitly and apply dependency vulnerability scanning in CI.
4. Pin each Hugging Face download to an immutable commit using the revision parameter.
5. Record and verify checksums for all downloaded model files before loading them.
6. Configure runtime loading for local-only operation after verification, for example by using supported local_files_only controls.
7. Fail closed when the verified local model is missing instead of silently downloading a replacement.
8. Document the initial network-dependent setup separately from offline runtime behavior.

T09 · Insecure Skill Coding Practices

Warning

• Location: USAGE.md:74

Finding

Shell Command Injection in Batch-Processing Example

Content

View full analysis

Vulnerability Details

File Location: USAGE.md:74-77
Vulnerability Type: Shell command injection through unsafe string interpolation
Risk Level: Medium

Vulnerable Code

python

# Process multiple folders
folders = ["./docs1", "./docs2", "./docs3"]
for folder in folders:
    os.system(f"python rag.py ingest --docs-path {folder}")

Technical Analysis

The example interpolates a folder path directly into a command string passed to os.system. This API invokes a system shell, so shell metacharacters in folder are interpreted as command syntax rather than as part of one path argument.

The example currently uses hardcoded folder names, so those exact values are not exploitable. However, it is presented as a reusable batch-processing pattern. If an implementation obtains folder values from command-line input, configuration, uploaded metadata, filenames, or another untrusted source, an attacker can append shell operators and arbitrary commands.

Quoting the value manually would remain fragile across shells and platforms. The correct defense is to avoid shell interpretation entirely.

Attack Path

1. A developer adopts the documented batch-processing example.
2. The folders collection is changed to include values from an untrusted source.
3. An attacker supplies a folder value containing shell metacharacters followed by a command.
4. The f-string constructs a single shell command containing both the intended ingestion command and the attacker-controlled command.
5. os.system passes the string to the operating-system shell.
6. The shell executes the injected command with the privileges of the RAG process.

Impact Assessment

Successful exploitation provides arbitrary command execution under the account running the batch-processing script. An attacker could read, modify, or delete accessible files; tamper with the RAG project or vector store; access documents available to the proce ...[truncated 230 chars]

Remediation

View remediation

Remediation Suggestions

Replace shell-based execution with an argument array:

python

import subprocess
import sys

folders = ["./docs1", "./docs2", "./docs3"]
for folder in folders:
    subprocess.run(
        [sys.executable, "rag.py", "ingest", "--docs-path", folder],
        check=True,
    )

Additionally:

1. Validate that each input resolves to an allowed directory.
2. Apply an explicit base-directory restriction when folders may be user-controlled.
3. Reject nonexistent paths and unexpected path types.
4. Do not enable shell=True in the replacement implementation.
5. Handle subprocess.CalledProcessError explicitly and log failures without exposing sensitive path information.

T09 · Insecure Skill Coding Practices

Note

• Location: USAGE.md:106

Finding

Flask Development Debugger Enabled in Web Interface Template

Content

View full analysis

Vulnerability Details

File Location: USAGE.md:106-107
Vulnerability Type: Unsafe debug configuration
Risk Level: Low

Vulnerable Code

python

if __name__ == '__main__':
    app.run(debug=True, port=5000)

Technical Analysis

The documented Flask application starts with development debug mode enabled. Debug mode can expose detailed exception pages containing stack traces, source excerpts, filesystem paths, and application state. Flask's development server and debugger are not designed for production deployment.

The shown call uses Flask's default loopback binding, which reduces immediate remote exposure. Risk arises when users adapt the example by changing the bind address, placing it behind a proxy, running it in a shared environment, or otherwise making the service reachable while retaining debug=True.

Attack Path

1. A user deploys or exposes the supplied web-interface template without disabling debug mode.
2. An attacker sends malformed requests or inputs that trigger an unhandled exception.
3. Flask returns a detailed debugger response.
4. The attacker extracts sensitive diagnostic information from the response.
5. If interactive debugger functionality is reachable and its protection is bypassed or misconfigured, the attacker may be able to execute Python code in the server process.

Impact Assessment

The direct impact includes disclosure of source code fragments, local paths, internal implementation details, and values present in exception context. In a remotely exposed and inadequately protected deployment, debugger abuse can potentially lead to arbitrary code execution with the privileges of the Flask service account.

In the exact default loopback-only example, exploitation generally requires local access or an additional exposure mechanism. This constraint lowers the finding's severity but does not make debug mode suitable for deployment.

Remediation

View remediation

Remediation Suggestions

1. Disable debug mode by default:

python

if __name__ == "__main__":
    app.run(debug=False, port=5000)

2. Keep development and production configuration separate.
3. Deploy through a production WSGI server rather than Flask's development server.
4. Apply authentication and authorization before exposing document-query endpoints.
5. Restrict network binding and firewall access to trusted clients.
6. Add centralized exception handling that returns generic errors while recording protected server-side logs.
7. Validate query and top_k request fields and impose request-size and rate limits before deployment.

SkillSpector

Vulnerability Patterns

• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• MCP Tool Poisoning: Hidden Instructions, Unicode Deception, Parameter Description Injection
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access
• Supply Chain: Unpinned Dependencies, External Script Fetching, Obfuscated Code

Findings (8)

Description-Behavior Mismatch

Medium

• Category: Not specified by scanner
• Confidence: 95% confidence

Finding

The README makes a strong 'complete offline' claim while the documented setup instructs users to download a model from Hugging Face, which requires network access and introduces a supply-chain dependency. This discrepancy can mislead users operating in air-gapped or security-sensitive environments into trusting the setup as offline-safe when it is not.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium

• Category: Not specified by scanner
• Confidence: 96% confidence

Finding

The skill explicitly advertises that it works 'completely offline,' but the provided embedding loader falls back to loading from Hugging Face when the local model is missing. In offline, privacy-sensitive, or regulated environments, this can silently trigger unexpected outbound network access and break security assumptions about isolation and data handling.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 94% confidence

Finding

The skill presents itself as offline, yet the documented model download step contacts Hugging Face and transmits network metadata such as IP address and request details. Without a clear warning, users may run this in environments where any external contact is prohibited or where privacy expectations require fully air-gapped behavior.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 95% confidence

Finding

The skill ingests documents and stores metadata plus chunk text to local disk, but the description does not clearly warn users that potentially sensitive document contents will be persisted. In a RAG context, this can expose confidential data through local files, backups, shared workstations, or improper filesystem permissions.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium

• Category: Not specified by scanner
• Confidence: 94% confidence

Finding

The manifest describes building and deploying local RAG systems with offline document processing, but the usage instructions explicitly rely on huggingface_hub.snapshot_download to fetch models from a remote service. While model acquisition can be a setup step, presenting network-dependent downloading as core usage conflicts with the manifest's offline emphasis.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 98% confidence

Finding

Running Flask with debug=True exposes the Werkzeug debugger, which can lead to sensitive information disclosure and, in some deployment scenarios, remote code execution if the service is reachable by untrusted users. In a skill intended to help users deploy a query interface, omitting any warning makes it more likely that users will run this unsafe configuration outside of a local-only development environment.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Low

• Category: Not specified by scanner
• Confidence: 88% confidence

Finding

The docstring describes 'local fallback,' but the implementation actually performs a remote Hugging Face load if the local path is unavailable. This mismatch is dangerous because operators may trust the code comments and deploy it in environments where any unexpected egress is a policy violation.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low

• Category: Not specified by scanner
• Confidence: 77% confidence

Finding

The usage guide instructs users to download models from Hugging Face via snapshot_download, which initiates an external network call. The document does not disclose that this step contacts a third-party service and transfers request metadata, which is a relevant privacy and network-behavior warning for markdown guidance.

Content

No source excerpt is available for this finding.