Back to skill

Security audit

Self-Improving Skill

Security checks for vulnerabilities and agentic risk

Overview

The skill is openly self-modifying, but its automatic mode can persist LLM-generated changes to other SKILL.md files while retaining and submitting user drafts without strong review or privacy controls.

Review this carefully before installing. Use it only on skills and drafts you are comfortable storing locally and potentially sending to your configured LLM provider. Avoid cron/auto mode unless you add human diff review, pin installation dependencies, restrict IMPROVE_LLM_CMD, and set retention/redaction rules for logs.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (3)

T01 · Skill Instruction Hijacking

Error
Location
scripts/improve.py:175
Finding

Prompt injection can persistently rewrite a target Skill

Content
View full analysis
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/observe.py:128
Finding

Sensitive document content is stored in plaintext and submitted to external-capable LLM clients

Content
View full analysis
Remediation
View remediation

T08 · Insecure Dependencies

Warning
Location
README.md:20
Finding

Installation guidance uses mutable and unpinned third-party dependencies

Content
View full analysis
Remediation
View remediation
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
  • Taint TrackingDirect Taint Flow, Variable-Mediated Taint Flow, Credential Exfiltration Chain
Findings (28)

Tp4

High
Category
MCP Tool Poisoning
Confidence
93% confidence
Finding

声明描述的是一个完整的“自我改进”系统:输入 original/final 后自动分析差异并把学习结果写回技能定义文件。实际代码仅实现了数据采集与管理部分:读取文本、生成短 hash、把 original/final 记录到日志、自动或按 hash 配对、列出未配对记录、展示统计信息。虽然这可作为后续学习系统的支撑层,但就该代码块本身而言,核心承诺的 diff、规则抽取、以及更新 SKILL.md 都缺失,因此描述与实际行为存在实质性不匹配。

Content

No source excerpt is available for this finding.

Lp1

High
Category
MCP Least Privilege
Confidence
93% confidence
Finding

The code has shell/process-execution capability via subprocess, but that capability is not transparently declared despite affecting local execution and data handling. In skill context, hidden execution capability is more dangerous because the stated purpose is content improvement, not running external tools chosen by environment.

Content

No source excerpt is available for this finding.

Lp1

High
Category
MCP Least Privilege
Confidence
93% confidence
Finding

The code has shell/process-execution capability via subprocess, but that capability is not transparently declared despite affecting local execution and data handling. In skill context, hidden execution capability is more dangerous because the stated purpose is content improvement, not running external tools chosen by environment.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
70% confidence
Finding

npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The cron-based auto flow describes automatic extraction and application of P0 rules to a target SKILL.md without a strong warning that a scheduled process may modify prompt/instruction files autonomously. In this context, automatic prompt mutation can cause integrity drift, unexpected behavior changes, or propagation of unsafe rules without timely human review.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The skill explicitly stores original drafts, final human-edited versions, and context in local logs, but does not prominently warn about privacy, retention, or sensitive-content handling. This can lead users to persist confidential business, personal, or regulated data in plaintext locations without informed consent or safeguards.

Content

No source excerpt is available for this finding.

Ssd 3

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The skill defines persistent logging of full AI-generated drafts, human-finalized content, and context into local JSONL files. This creates a durable data collection channel that can accumulate sensitive prompts, proprietary text, secrets, and personal data, increasing exposure through local compromise, backups, syncing, or accidental sharing.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
91% confidence
Finding

The skill instructs users to run npx clawhub@latest install self-improving-skill, which fetches and executes the latest package version at install time without pinning or integrity controls. That creates a supply-chain risk: a compromised upstream package or malicious new release could execute arbitrary code on the user's machine.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
98% confidence
Finding

The module docstring and command/help text are written in Chinese, and the prompts sent to the LLM are also Chinese-only. This imposes a specific language/locale without opt-in or explanation, which matches the policy category for language or locale constraints.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

A writing-style self-improvement skill mainly needs to read logs, analyze diffs, and update SKILL.md. Spawning external binaries—especially a user-specified command—introduces a broader execution capability that is not inherently required by that purpose as described in the manifest.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The script sends current SKILL.md content and human edit logs to external LLM CLIs without explicit notice, consent, redaction, or provider-boundary disclosure. Those logs may contain sensitive user text, internal guidance, or proprietary content, making silent exfiltration a significant privacy and compliance risk.

Content

No source excerpt is available for this finding.

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · scripts/improve.py (reported line 132)May include surrounding context.

python
]
    for cmd in candidates:
        try:
            result = subprocess.run(cmd, input=prompt, capture_output=True,
                                    text=True, timeout=timeout)
            if result.returncode == 0 and result.stdout.strip():
                return result.stdout.strip()

Tainted flow: 'prompt' from os.environ.get (line 296, credential/environment) → subprocess.run (code execution)

Medium
Category
Data Flow
Confidence
65% confidence
Finding

Data from a source is assigned to a variable that is later passed to a sink, creating a variable-mediated taint flow.

Content

Scanner excerpt · scripts/improve.py (reported line 132)May include surrounding context.

python
]
    for cmd in candidates:
        try:
            result = subprocess.run(cmd, input=prompt, capture_output=True,
                                    text=True, timeout=timeout)
            if result.returncode == 0 and result.stdout.strip():
                return result.stdout.strip()

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The skill is presented as a writing-style improver, but it includes a configurable external command execution path that can run arbitrary binaries. This mismatch increases deception risk and can hide dangerous capability behind an innocuous description.

Content

No source excerpt is available for this finding.

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
96% confidence
Finding

The command is taken from the IMPROVE_LLM_CMD environment variable and executed as a process, allowing an attacker who can influence the environment to run an arbitrary binary under the user's privileges. Although split() avoids shell metacharacter expansion, it still permits arbitrary program execution and unsafe argument parsing.

Content

Scanner excerpt · scripts/improve.py (reported line 143)May include surrounding context.

python
custom_cmd = os.environ.get("IMPROVE_LLM_CMD")
    if custom_cmd:
        try:
            result = subprocess.run(custom_cmd.split(), input=prompt,
                                    capture_output=True, text=True, timeout=timeout)
            if result.returncode == 0 and result.stdout.strip():
                return result.stdout.strip()

Tainted flow: 'prompt' from os.environ.get (line 296, credential/environment) → subprocess.run (code execution)

Medium
Category
Data Flow
Confidence
95% confidence
Finding

This sink combines untrusted prompt content with an arbitrarily configurable executable from IMPROVE_LLM_CMD, creating a genuine arbitrary-program-execution capability. While the prompt itself is stdin data, the environment-controlled command makes the overall flow dangerous in a multi-user or compromised execution environment.

Content

Scanner excerpt · scripts/improve.py (reported line 143)May include surrounding context.

python
custom_cmd = os.environ.get("IMPROVE_LLM_CMD")
    if custom_cmd:
        try:
            result = subprocess.run(custom_cmd.split(), input=prompt,
                                    capture_output=True, text=True, timeout=timeout)
            if result.returncode == 0 and result.stdout.strip():
                return result.stdout.strip()

Ssd 3

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The prompt includes up to 3000 characters each of original and final human-edited content plus contextual metadata and part of the current SKILL.md, all of which may be sensitive. Sending these materials to an external model can expose private user content and internal operational rules beyond what is necessary for rule extraction.

Content

No source excerpt is available for this finding.

Tainted flow: 'backup_path' from os.environ.get (line 269, credential/environment) → shutil.copy2 (file write)

Medium
Category
Data Flow
Confidence
65% confidence
Finding

Data from a source is assigned to a variable that is later passed to a sink, creating a variable-mediated taint flow.

Content

Scanner excerpt · scripts/improve.py (reported line 270)May include surrounding context.

python
return None
    name = f"SKILL-{datetime.now().strftime('%Y%m%d-%H%M%S')}.md"
    backup_path = backup_dir / name
    shutil.copy2(target, backup_path)
    print(f"📦 备份: {backup_path}")
    return backup_path

Ssd 3

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The merge step transmits the full proposal and the complete current SKILL.md to the model and asks it to regenerate the entire file, which can disclose sensitive policy content and amplify prompt injection risks. Regenerating the whole file also increases the blast radius of hallucinations or malicious inserted instructions.

Content

No source excerpt is available for this finding.

Tainted flow: 'updated' from os.environ.get (line 312, credential/environment) → pathlib.Path.write_text (file write)

Medium
Category
Data Flow
Confidence
94% confidence
Finding

The script writes untrusted LLM output directly into the target SKILL.md, enabling prompt-injection-driven persistence of malicious instructions or corruption of policy files. In auto mode this becomes unattended modification, increasing the chance of silent compromise of future agent behavior.

Content

Scanner excerpt · scripts/improve.py (reported line 317)May include surrounding context.

python
print("❌ 合并失败")
        return

    target.write_text(updated)

    new_content = proposal_content.replace(
        "status: pending",

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

Auto mode performs unattended extraction and application of P0 rules to SKILL.md, allowing persistent policy changes without human verification. Because the source material and merge logic are LLM-driven, prompt injection or misclassification can silently poison future outputs.

Content

No source excerpt is available for this finding.

Tainted flow: 'emergency' from os.environ.get (line 374, credential/environment) → shutil.copy2 (file write)

Medium
Category
Data Flow
Confidence
65% confidence
Finding

Data from a source is assigned to a variable that is later passed to a sink, creating a variable-mediated taint flow.

Content

Scanner excerpt · scripts/improve.py (reported line 375)May include surrounding context.

python
# 保存当前版本
    if target.exists():
        emergency = backup_dir / f"SKILL-pre-rollback-{datetime.now().strftime('%Y%m%d-%H%M%S')}.md"
        shutil.copy2(target, emergency)

    shutil.copy2(latest, target)
    print(f"✅ 已回滚到: {latest.name}")

Tainted flow: 'latest' from os.environ.get (line 371, credential/environment) → shutil.copy2 (file write)

Medium
Category
Data Flow
Confidence
65% confidence
Finding

Data from a source is assigned to a variable that is later passed to a sink, creating a variable-mediated taint flow.

Content

Scanner excerpt · scripts/improve.py (reported line 377)May include surrounding context.

python
emergency = backup_dir / f"SKILL-pre-rollback-{datetime.now().strftime('%Y%m%d-%H%M%S')}.md"
        shutil.copy2(target, emergency)

    shutil.copy2(latest, target)
    print(f"✅ 已回滚到: {latest.name}")

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The module docstring, usage notes, help text, and console messages are written in Chinese only, which imposes a language choice on users without offering an alternative. This can violate language/locale policy where user-facing language should be selectable or explicitly justified.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The script writes the full original content to local JSONL logs, including potentially sensitive user text, with no minimization, consent prompt, retention control, or redaction. In a self-improving writing skill, inputs may contain drafts, proprietary material, credentials pasted by mistake, or personal data, so persistent plaintext storage materially increases confidentiality risk if the machine, backup system, or shared home directory is accessed.

Content

No source excerpt is available for this finding.