Back to skill

Security audit

Improvement Generator

Security checks for vulnerabilities and agentic risk

Overview

The skill is mostly coherent, but it can send target skill content to an external Claude CLI and mark model-generated SKILL.md changes as low-risk and executor-supported.

Install only if you are comfortable with candidate generation that may call a local Claude CLI and send target SKILL.md plus evaluator failure details to it. Treat any generated SKILL.md change as requiring human review, especially because this version labels evaluator-driven prompt fixes as low-risk and executor-supported despite their ability to change future agent behavior.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Findings (1)

T01 · Skill Instruction Hijacking

Warning
Location
scripts/propose.py:286
Finding

Untrusted Content Can Influence an Auto-Executable Skill Modification

Content
View full analysis
", ' '"action": "append_markdown_section" or "replace_markdown_section", ' '"content_lines": ["line1", "line2", ...], ' '"rationale": "why this fixes the failures"}}' ) try: result = subprocess.run( ["claude", "-p", "--output-format", "json"], input=prompt, capture_output=True, text=True, timeout=120, ) if result.returncode != 0: return None # Parse claude output try: claude_out = _json.loads(result.stdout) text = claude_out.get("result", result.stdout) except (_json.JSONDecodeError, TypeError): text = result.stdout # Extract JSON from response (may be wrapped in markdown) text = text.strip() # Find the first { and last } to extract JSON robustly first_brace = text.find("{") last_brace = text.rfind("}") if first_brace >= 0 and last_brace > first_brace: text = text[first_brace:last_brace + 1] parsed = _json.loads(text) return { "id": "cand-01-eval-fix", "title": f"Fix SKILL.md based on {len(failed_tasks)} evaluator failure(s)", "target_path": str(skill_md.resolve()), ...[truncated 3894 chars]
Remediation
View remediation
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Findings (7)

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
91% confidence
Finding

The skill advertises and documents behaviors that imply shell execution, file reads/writes, and networked LLM calls, but it does not declare any explicit tool scope or permission boundaries. That makes the runtime trust model ambiguous and can allow the skill to be executed with broader capabilities than reviewers or orchestrators expect, increasing the chance of unsafe file modification, command execution, or data exfiltration through external model calls.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The skill is presented as an improvement-candidate generator, but this function goes further by reading target content and synthesizing concrete fixes via an external LLM. That mismatch increases risk because users may expect local analysis only, while sensitive target material and failure traces are actually transmitted to another tool/service for modification guidance.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

Calling claude is not necessary for the baseline purpose of proposing improvement candidates and introduces an unnecessary external execution path. In a skill that processes potentially adversarial target content, this broadens the attack surface through prompt injection, unreviewed content export, and environment-dependent behavior.

Content

No source excerpt is available for this finding.

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
92% confidence
Finding

The code invokes an external claude CLI with target SKILL.md content and evaluator failure details, which creates an execution boundary and a data-exfiltration surface outside the current process. Although arguments are passed as a list and there is no shell injection here, untrusted repository content is still forwarded to an external model/tool without clear trust, sandboxing, or consent controls.

Content

Scanner excerpt · scripts/propose.py (reported line 304)May include surrounding context.

python
)

    try:
        result = subprocess.run(
            ["claude", "-p", "--output-format", "json"],
            input=prompt, capture_output=True, text=True, timeout=120,
        )

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
87% confidence
Finding

The subprocess call to an external LLM CLI occurs without any user-facing disclosure, even though the prompt includes target SKILL.md content and evaluator failure details. Lack of transparency is dangerous because operators may unknowingly expose proprietary or sensitive repository data to external tooling or remote services.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
89% confidence
Finding

Several user-facing title/rationale strings are hardcoded in Chinese while other output strings are in English, implying a fixed language choice rather than offering locale selection. This can violate language/locale policy when the skill does not document that it is Chinese-only or provide a user opt-in for that language.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Low
Category
Not specified by scanner
Confidence
78% confidence
Finding

The docstring states candidate priorities are adjusted based on the failure trace, and inline comments say the failed category is deprioritized while alternatives are boosted. In practice, alternatives are inserted at the front on each iteration and failed-category items are merely appended, so the behavior is not a clear implementation of the documented prioritization intent and can contradict the stated ordering semantics.

Content

No source excerpt is available for this finding.