Back to skill

Security audit

Smart Model Selector

Security checks for vulnerabilities and agentic risk

Overview

The skill is a coherent model router, but it needs review because rated sessions can persist full prompt text in a local plaintext database without clear opt-in or retention controls.

Install only if you are comfortable with a local database of rated task prompts and metadata. Avoid using feedback/rating commands on sensitive chats, periodically delete the documented model_selection.db if you do not want history retained, and prefer a version that stores derived routing features instead of full prompt text or offers a no-logging mode.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Warning
Location
src/model_selector.py:108
Finding
Plaintext Persistence of Complete User Prompts Without Data Minimization<![CDATA[ ## Vulnerability Details **File Location**: `hooks/smart_model_selector.py:35-48`; `src/model_selector.py:61-77`; `src/model_selector.py:108-128`; `src/model_selector.py:374-382`; `src/model_selector.py:402-407` **Vulnerability Type**: T09: Insecure Skill Coding Practices **Risk Level**: Medium ### Vulnerable Code ```python # hooks/smart_model_selector.py:35-48 # Get the user's first message, if present first_message = bootstrap_info.get('first_message', '') if not first_message: return {'action': 'continue'} # Select the optimal model model, reason = selector.select_model(first_message) # Start tracking the task selector.start_task(first_message, model) ``` ```python # src/model_selector.py:61-77 c.execute(''' CREATE TABLE IF NOT EXISTS task_records ( id INTEGER PRIMARY KEY AUTOINCREMENT, task_hash TEXT UNIQUE NOT NULL, task_text TEXT NOT NULL, selected_model TEXT NOT NULL, dialogue_rounds INTEGER DEFAULT 1, user_rating INTEGER, duration_seconds REAL DEFAULT 0, token_consumption INTEGER DEFAULT 0, is_completed INTEGER DEFAULT 0, score REAL DEFAULT 0, created_at TEXT, updated_at TEXT ) ''') ``` ```python # src/model_selector.py:108-128 c.execute(''' INSERT OR REPLACE INTO task_records (task_hash, task_text, selected_model, dialogue_rounds, user_rating, duration_seconds, token_consumption, is_completed, score, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) ''', ( record.task_hash, record.task_text, record.selected_model, record.dialogue_rounds, record.user_rating, record.duration_seconds, record.token_consumption, 1 if record.is_completed else 0, self._calculate_score(record), record.created_at, record.updated_at )) conn.commit() conn.close() ``` ```python # src/model_selector.py:374-382 def start_task(self, task_text: str, selected_model: str = None): ...[truncated 3988 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. **Do not persist complete prompts by default** - Store only the selected model, derived task category, quality score, message-length bucket, and other non-content metadata needed for routing. - Remove the `task_text` column or replace it with minimized derived features. 2. **Use a non-reversible identifier** - If duplicate detection is required, use an HMAC with a locally generated secret instead of retaining the original text. - Do not treat unsalted MD5 as a privacy mechanism; predictable prompts can be recovered through dictionary guessing. 3. **Require explicit opt-in consent** - Disable learning-related persistence by default. - Clearly disclose which fields are stored, where they are stored, what event triggers persistence, and how long they are retained. 4. **Redact sensitive information** - Before any optional persistence, remove common credentials, authorization headers, private keys, connection strings, email addresses, and other sensitive patterns. - Warn users that automated redaction cannot guarantee removal of all confidential information. 5. **Protect retained data** - If storing prompt text is strictly necessary, encrypt the database or sensitive columns using a key held outside the database. - Create the data directory and database with owner-only permissions, such as mode `0700` for the directory and `0600` for the database, where supported. 6. **Implement retention and deletion controls** - Add configurable expiry and automatic deletion of old records. - Provide a documented command that securely removes all stored learning data. - Avoid retaining abandoned in-memory tasks longer than the active session. 7. **Align documentation with actual behavior** - Explain that `/model-rate` currently triggers persistence of the complete first message. - Update privacy claims to describe plaintext local storage accurately, or change the implementation to satisfy strong ...[truncated 26 chars]
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (15)

Missing User Warnings

Medium
Confidence
90% confidence
Finding
The README explicitly advertises persistent recording of conversation outcomes and local storage, but does not clearly warn users what data is retained, for how long, or whether prompts and derived metadata are stored. In a model-routing skill, this can expose sensitive prompt content or behavioral history through local files or later unintended access, making the privacy risk real even if storage is only local.

Missing User Warnings

Medium
Confidence
92% confidence
Finding
The README says the skill activates automatically and analyzes task content, but does not clearly inform users that their prompts will be inspected for routing decisions. This is a meaningful transparency and consent issue because users may assume they are talking directly to the model, not having their content pre-processed and classified by a separate routing component.

Natural-Language Policy Violations

Medium
Confidence
84% confidence
Finding
The manifest description is entirely in Chinese, which imposes a specific language presentation in a user-facing field without any opt-in or explanation that the skill is intended only for Chinese-speaking users. Under the language/locale policy, user-facing language constraints should either be optional or clearly justified.

Vague Triggers

Medium
Confidence
90% confidence
Finding
The skill states it is 'automatically activated' for conversations without defining scope, opt-in boundaries, or exclusions. For a model-routing skill that inspects task content and records outcomes, broad automatic activation increases the chance that all prompts are processed and logged unexpectedly, creating privacy and user-consent risks.

Vague Triggers

Medium
Confidence
91% confidence
Finding
The manifest auto-registers the hook for broad lifecycle events including session.start, message.received, response.sent, and session.end, with no visible trigger constraints or scoping. For a model-routing skill, this creates unnecessary access to nearly all user prompts and model outputs, increasing the blast radius for data exposure, unintended prompt interception, and abuse if the hook logic is flawed or later modified maliciously.

Natural-Language Policy Violations

Medium
Confidence
81% confidence
Finding
The natural-language description is written only in Chinese and does not indicate that the user can choose another language or that the skill is intentionally limited to a Chinese-speaking context. This can violate language/locale policy when a skill implicitly forces a specific language without opt-in.

Vague Triggers

Medium
Confidence
88% confidence
Finding
The manifest subscribes to broad lifecycle events ("agent:bootstrap" and "command") without any documented trigger constraints, which causes the skill to load and potentially run in many contexts beyond narrowly intended use. In a model-routing skill, this increases the attack surface because routing logic can influence requests globally at startup and across arbitrary commands, making unintended interception or policy bypass more likely if the implementation is flawed or later modified.

Natural-Language Policy Violations

Medium
Confidence
95% confidence
Finding
The module-level description states in Chinese that the hook automatically activates at every session start, and all user-facing strings in the file are Chinese-only. This creates a language/locale constraint without any opt-in or alternative, which matches the policy category for forced language behavior.

Natural-Language Policy Violations

Medium
Confidence
88% confidence
Finding
The file-level natural-language description is written entirely in Chinese and presents the skill as a Chinese-localized tool without indicating language choice or user opt-in. Under the policy, forcing a specific language or locale without offering a choice is a natural-language policy concern.

Description-Behavior Mismatch

Medium
Confidence
94% confidence
Finding
The code initializes a persistent SQLite database that stores full task text, model choice, ratings, timing, token usage, and completion metadata. For a model-routing helper, retaining raw prompts and user telemetry creates unnecessary data exposure and privacy risk if the local database is accessed by other tools, users, or malware on the host.

Context-Inappropriate Capability

Medium
Confidence
95% confidence
Finding
The save path persistently writes raw task text and feedback-derived telemetry into local storage without any access controls, encryption, or data minimization. Because task text may contain secrets, proprietary code, credentials, or personal data, this expands the skill from routing into silent prompt logging and increases the consequences of local compromise.

Missing User Warnings

Medium
Confidence
95% confidence
Finding
Task text is saved persistently with no user-facing warning or indication that prompts and feedback are being recorded. This is dangerous because users may enter sensitive information under the assumption it is transient, causing unanticipated privacy, compliance, and data-handling risks.

Natural-Language Policy Violations

Low
Confidence
73% confidence
Finding
The prominent natural-language description and examples are entirely in Chinese, and the README does not indicate whether the skill supports other languages or whether Chinese is a deliberate locale restriction. Under SQP-3, forcing or implying a specific language without user opt-in can be a policy concern unless clearly justified.

Missing User Warnings

Low
Confidence
85% confidence
Finding
The introductory description emphasizes that the system 'learns' and records conversation effectiveness, but the early description does not clearly warn that conversation-derived metadata and user ratings are retained locally. This can mislead users about persistence of their interaction data and reduce informed consent, even if storage is only local.

Natural-Language Policy Violations

Low
Confidence
86% confidence
Finding
The manifest description uses Chinese-language text, but the file does not indicate that the skill is intended only for Chinese-speaking users or provide any language/locale opt-in. This can conflict with organizational language-choice expectations when users are not given a documented option.

Static analysis

No suspicious patterns detected.