Security audit

Modelsense

Security checks for vulnerabilities and agentic risk

Overview

ModelSense is mostly a model-recommendation skill, but one included agent instruction can proactively invoke it and switch models without explicit user consent.

Review this skill before installing. It is reasonable as an advisory model picker, but remove or override AGENTS.md unless you want agents to invoke ModelSense early in sessions and possibly switch models after only notifying you. Require explicit confirmation before any model switch or delegation, and audit the GitHub Actions workflow if you maintain the package repository.

A.I.G

Vulnerability Patterns

Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
Insecure DependenciesIntroduces malicious components through unsafe dependency sources
Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally

Findings (2)

T01 · Skill Instruction Hijacking

Error

Location: AGENTS.md:3
Finding: Unsolicited Skill Invocation and Model Switching Without Explicit Consent
Content: View full analysis
Remediation: View remediation

T08 · Insecure Dependencies

Warning

Location: .github/workflows/update-models.yml:13
Finding: Unpinned CI Dependencies Execute with Repository Write Access and Secret Availability
Content: View full analysis
Remediation: View remediation
- uses: actions/setup-python@ ``` 2. Create a locked requirements file containing exact package versions and cryptographic hashes. 3. Install dependencies using hash verification: ```yaml - name: Install dependencies run: python -m pip install --require-hashes -r requirements.txt ``` 4. Use a dependency-update service to propose reviewed updates rather than resolving unconstrained versions during each run. 5. Determine whether the public OpenRouter model endpoint requires authentication. If it does not, remove `OPENROUTER_API_KEY` from the workflow entirely. 6. If authentication is necessary, use a dedicated read-only or minimum-scope key with strict quota and rotation controls. 7. Consider separating data generation from repository writes. For example, generate and validate an artifact in a read-only job, then permit a separate narrowly controlled job to commit only the expected file. 8. Validate the generated YAML schema and ensure that only the intended pricing, context-window, and metadata fields changed before pushing. 9. Protect the target branch with review requirements or have the workflow open a pull request instead of pushing directly. ]]>

SkillSpector

Vulnerability Patterns

Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration

Findings (7)

Tp4

High

Category: MCP Tool Poisoning
Confidence: 99% confidence
Finding: The declared purpose is a user-facing model recommendation skill, but the finding indicates behavior tied to external API use and local data writing, which is materially different from the advertised function. Description/behavior mismatches are dangerous because they defeat user and reviewer expectations, hide higher-risk operations, and can enable unauthorized data access or persistence under the cover of a benign-seeming skill.
Content: No source excerpt is available for this finding.

Vague Triggers

Medium

Category: Not specified by scanner
Confidence: 92% confidence
Finding: The invocation examples are broad, natural-language phrases that overlap with common user requests, increasing the chance this skill is triggered when the user did not specifically intend to invoke it. In a skill system, overly generic triggers can cause accidental routing into a capability that may then recommend or initiate model changes.
Content: No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium

Category: Not specified by scanner
Confidence: 92% confidence
Finding: The README presents the skill as advisory, but then states it can auto-switch the session model or delegate to a sub-agent. That changes the trust boundary from recommendation to action, and if invoked implicitly or without explicit user confirmation it could alter execution behavior, cost, data exposure, or downstream agent choice in ways the user did not intend.
Content: No source excerpt is available for this finding.

Missing User Warnings

Medium

Category: Not specified by scanner
Confidence: 95% confidence
Finding: The README describes automatic model switching and delegation but provides no warning about user-visible consequences such as increased cost, different provider selection, data sharing with another model/provider, or changes in agent behavior. In this context, omission of consent and impact disclosure makes the feature more dangerous because the skill sits in a decision-making path for model/provider use.
Content: No source excerpt is available for this finding.

Undeclared Tool Scope

Medium

Category: MCP Least Privilege
Confidence: 89% confidence
Finding: The skill text instructs use of external execution capabilities such as openclaw models list and session-changing actions, but it declares no explicit tool scope or permissions boundaries. That makes the effective authority of the skill ambiguous and can lead to overbroad tool access, especially if the runtime infers or grants tools implicitly.
Content: No source excerpt is available for this finding.

Vague Triggers

Medium

Category: Not specified by scanner
Confidence: 91% confidence
Finding: The trigger bullets 'User wants to understand what a benchmark means' and 'User wants ModelSense to auto-switch the session model' are broad and could overlap with many general conversations or other skills. The file does not provide negative examples or narrower scope boundaries for when these requests should invoke this specific skill versus a general assistant response.
Content: No source excerpt is available for this finding.

Natural-Language Policy Violations

Low

Category: Not specified by scanner
Confidence: 84% confidence
Finding: The entry recommends DeepSeek Chat for "Chinese language tasks," which is a natural-language locale preference embedded in a manifest file. Because the file does not indicate user choice, opt-in, or a documented region-specific constraint, this can be interpreted as a locale-policy issue under the rule for language/locale violations.
Content: No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.