Back to skill

Security audit

semantic-model-router

Security checks for vulnerabilities and agentic risk

Overview

The router is mostly coherent, but it silently saves raw user prompts locally and has under-disclosed model/dependency retrieval risks, so users should review it before installing.

Install only if you are comfortable with routed prompts being saved locally in plaintext unless the code is changed or configured to avoid it. Avoid sending secrets, proprietary code, customer data, or regulated information through this router until logging is disabled, the storage path and deletion process are documented, and dependencies/model artifacts are pinned or otherwise verified.

Vulnerability Patterns
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (2)

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/model_router.py:293
Finding
Full User Queries Are Silently Persisted in Plaintext<![CDATA[ ## Vulnerability Details **File Location**: `scripts/model_router.py`, lines 293 and 384-398 **Vulnerability Type**: Plaintext storage of potentially sensitive prompt data **Risk Level**: Medium ### Vulnerable Code ```python self._log_query(query, best_tier) ``` ```python def _log_query(self, query: str, tier: str) -> None: """Append the query + tier to the history file for offline analysis.""" try: history: list = [] if os.path.exists(self.history_file): with open(self.history_file, "r", encoding="utf-8") as f: raw = f.read().strip() if raw: history = json.loads(raw) history.append({"query": query, "tier": tier}) if len(history) > 1000: history = history[-1000:] with open(self.history_file, "w", encoding="utf-8") as f: json.dump(history, f, ensure_ascii=False, indent=2) except Exception: pass ``` ### Technical Analysis Every call to `ModelRouter.route()` invokes `_log_query()`, which stores the complete user query and its assigned tier in `query_history.json` by default. Logging is not disabled by default and does not require user consent. The history file can retain up to 1,000 complete prompts. No secret filtering, data minimization, encryption, explicit file permissions, or secure application-data location is used. Because the default path is relative, the file is created in the process working directory and may be exposed through source-control commits, shared workspaces, backups, build artifacts, or other local processes. Prompts commonly contain proprietary source code, credentials, personal information, internal architecture details, or confidential business data. Silently retaining this content violates data-minimization and secure-storage principles. The broad exception handler also conceals logging and permission failures, making the behavior difficult to monitor or audit. ### Attack Pa ...[truncated 1278 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Disable query-content logging by default and require explicit user opt-in. 2. Store only necessary derived metadata, such as an anonymous identifier, routing tier, and aggregate statistics. 3. Never retain raw prompts unless the user explicitly enables diagnostic logging. 4. Apply secret and personal-data redaction before writing any diagnostic records. 5. Store logs in a dedicated private application-data directory rather than the current working directory. 6. Create the history file with owner-only permissions, such as mode `0600` on POSIX systems. 7. Provide configurable retention limits and automatic expiration. 8. Document the logging behavior, file location, retained fields, and deletion procedure. 9. Replace the silent exception handler with controlled diagnostic reporting that does not expose prompt content. 10. Consider an explicit constructor option such as: ```python def __init__(self, ..., enable_history: bool = False): self.enable_history = enable_history ``` Then conditionally invoke logging: ```python if self.enable_history: self._log_query(query, best_tier) ``` ]]>

T08 · Insecure Dependencies

Note
Location
scripts/model_router.py:206
Finding
Runtime Model Retrieval and Unbounded Dependency Versions Create Supply-Chain Exposure<![CDATA[ ## Vulnerability Details **File Location**: `scripts/model_router.py`, line 206; `scripts/requirements.txt`, lines 1-2; `SKILL.md`, lines 141-143 **Vulnerability Type**: Unpinned dependencies and implicit retrieval of external model artifacts **Risk Level**: Low ### Vulnerable Code `scripts/model_router.py`: ```python self.encoder = SentenceTransformer("all-MiniLM-L6-v2") ``` `scripts/requirements.txt`: ```text sentence-transformers>=2.2.2 numpy>=1.24.0 ``` Relevant documentation in `SKILL.md`: ```text ## Security & Privacy - **Zero external calls**: All classification runs locally. - **No API keys**: The router itself needs none. - **Transparent weights**: All model parameters live in `scripts/model_weights.py` — fully auditable. ``` ### Technical Analysis `SentenceTransformer("all-MiniLM-L6-v2")` can resolve and download model files from an external model registry when the requested model is not already available in the local cache. Although inference subsequently runs locally, first-time initialization can therefore perform external network access. The model identifier is not bound to an immutable revision or locally verified artifact. The effective model files may consequently depend on the state of the external registry, local cache configuration, and installed library behavior at runtime. The dependency declarations use open-ended minimum versions. A later installation can retrieve versions substantially newer than those reviewed by the project. No exact versions, integrity hashes, lock file, or upper bounds are supplied. This prevents reproducible installation and increases exposure to compromised releases or incompatible future behavior. The runtime retrieval behavior also conflicts with the unconditional documentation claim that the Skill makes “Zero external calls.” The audit found no evidence that the current dependencies or model are malicious; this finding concerns avoidable supply-chain exposure and inaccurate disclosure. ...[truncated 1578 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Pin all direct dependencies to reviewed exact versions rather than open-ended minimum versions. 2. Generate a lock file that includes the complete transitive dependency graph. 3. Use package hashes, such as pip `--require-hashes`, to verify downloaded distributions. 4. Download and review model files during a controlled build or installation step. 5. Pin the sentence-transformer model to a specific immutable revision. 6. Record and verify cryptographic hashes for all bundled or downloaded model artifacts. 7. Bundle the reviewed model files when package size and licensing permit. 8. Configure runtime initialization to load only local files and fail closed if assets are unavailable. 9. Separate optional semantic dependencies from the minimal keyword-routing installation. 10. Update the documentation to state clearly whether first-time initialization may access an external registry. 11. Perform automated dependency and artifact vulnerability scanning as part of releases. 12. Avoid loading mutable serialized formats from untrusted sources and restrict any future `joblib` classifier file to package-controlled, integrity-verified content. ]]>
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Findings (12)

Tp4

High
Category
MCP Tool Poisoning
Confidence
95% confidence
Finding
This is a meaningful description-behavior mismatch because the skill is marketed as a local router using a pre-trained classifier with no extra API keys, but the analysis indicates additional behaviors including undeclared query logging/history persistence and broader routing logic than advertised. Hidden persistence of user prompts can expose sensitive data, and inaccurate claims about how routing works can mislead operators about privacy, cost, and model-selection behavior.

Tp4

High
Category
MCP Tool Poisoning
Confidence
95% confidence
Finding
This is a meaningful description-behavior mismatch because the skill is marketed as a local router using a pre-trained classifier with no extra API keys, but the analysis indicates additional behaviors including undeclared query logging/history persistence and broader routing logic than advertised. Hidden persistence of user prompts can expose sensitive data, and inaccurate claims about how routing works can mislead operators about privacy, cost, and model-selection behavior.

Lp3

Medium
Category
MCP Least Privilege
Confidence
84% confidence
Finding
The skill metadata does not declare any tool scope or permissions, yet the implementation reportedly performs file read/write operations. That creates an under-declared capability boundary: users may install or trust the skill based on incomplete metadata while the code can still access or persist local data, which is especially concerning given the noted query history persistence.

Insecure deserialization: joblib.load()

Medium
Category
Dangerous Code Execution
Content
# Priority: binary joblib > text weights > semantic embeddings
                model_path = os.path.join(os.path.dirname(__file__), "classifier.joblib")
                if os.path.exists(model_path):
                    self.classifier = joblib.load(model_path)
                elif _HAS_WEIGHTS:
                    self.classifier = self._create_manual_classifier()
                else:
Confidence
97% confidence
Finding
The code deserializes a local `classifier.joblib` file with `joblib.load()`, which uses pickle semantics and can execute attacker-controlled code during loading. If an attacker can replace or plant that file in the skill directory or supply-chain path, code execution occurs at router initialization before any validation.

Description-Behavior Mismatch

Medium
Confidence
96% confidence
Finding
The skill advertises routing functionality but also stores full user queries persistently in `query_history.json`, which expands its data handling beyond the stated purpose. Because queries may contain secrets, personal data, code, or business information, this creates an avoidable confidentiality and retention risk.

Missing User Warnings

Medium
Confidence
95% confidence
Finding
Every call to `route()` triggers `_log_query()` without any user-facing notice or confirmation, so sensitive prompts are silently written to disk. Silent persistence undermines user expectations and can expose confidential input that users believed was transient.

Ssd 3

Medium
Confidence
96% confidence
Finding
The code keeps up to 1000 prior queries in plain language for offline analysis, creating a durable store of potentially sensitive natural-language data. In a routing skill, prompts are especially likely to include user tasks, internal code, API details, or confidential business text, so retention materially increases leakage impact.

Context-Inappropriate Capability

Medium
Confidence
96% confidence
Finding
Persistently collecting raw query text is not necessary for making a per-request routing decision, so the storage is excessive relative to the stated function. Retaining natural-language prompts increases exposure of sensitive user content to local compromise, backups, debugging access, or later unintended reuse.

Missing User Warnings

Medium
Confidence
97% confidence
Finding
The `_log_query()` implementation writes the full query text and assigned tier to a local JSON file in plaintext, with no visible runtime disclosure. Plaintext storage of arbitrary user prompts can capture credentials, tokens, proprietary code, or personal information and make later leakage more likely.

Unpinned Dependencies

Low
Category
Supply Chain
Content
sentence-transformers>=2.2.2
numpy>=1.24.0
Confidence
94% confidence
Finding
The dependency is specified with a lower bound only, which allows future installs to resolve to different versions over time. This creates a supply-chain and reproducibility risk because a newly released vulnerable or incompatible version of sentence-transformers could be pulled into the environment without review.

Unpinned Dependencies

Low
Category
Supply Chain
Content
sentence-transformers>=2.2.2
numpy>=1.24.0
Confidence
97% confidence
Finding
numpy is also unpinned, so installations are not reproducible and may silently consume newly published releases. In a skill that relies on ML libraries, this increases supply-chain exposure and the chance of pulling a vulnerable or breaking version during deployment.

Unverifiable Dependency: numpy has 16 known advisory(ies) (CVE-2014-1859 (Numpy arbitrary file write via symlink attack); CVE-2021-41495 (NumPy NULL Pointer Dereference); CVE-2021-33430 (NumPy Buffer Overflow (Disputed)) +13 more), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low
Category
Supply Chain
Confidence
88% confidence
Finding
The manifest includes numpy without a pinned version, and numpy has multiple historical advisories. Because the resolved version is unknown, there is no assurance that deployments will avoid affected releases, making the dependency posture unverifiable and increasing supply-chain uncertainty.

Static analysis

No suspicious patterns detected.