T01 · Skill Instruction Hijacking
- Location
scripts/improve.py:175- Finding
Prompt injection can persistently rewrite a target Skill
- Content
View full analysis
- Remediation
View remediation
Security audit
Security checks for vulnerabilities and agentic risk
The skill is openly self-modifying, but its automatic mode can persist LLM-generated changes to other SKILL.md files while retaining and submitting user drafts without strong review or privacy controls.
Review this carefully before installing. Use it only on skills and drafts you are comfortable storing locally and potentially sending to your configured LLM provider. Avoid cron/auto mode unless you add human diff review, pin installation dependencies, restrict IMPROVE_LLM_CMD, and set retention/redaction rules for logs.
scripts/improve.py:175Prompt injection can persistently rewrite a target Skill
scripts/observe.py:128Sensitive document content is stored in plaintext and submitted to external-capable LLM clients
README.md:20Installation guidance uses mutable and unpinned third-party dependencies
声明描述的是一个完整的“自我改进”系统:输入 original/final 后自动分析差异并把学习结果写回技能定义文件。实际代码仅实现了数据采集与管理部分:读取文本、生成短 hash、把 original/final 记录到日志、自动或按 hash 配对、列出未配对记录、展示统计信息。虽然这可作为后续学习系统的支撑层,但就该代码块本身而言,核心承诺的 diff、规则抽取、以及更新 SKILL.md 都缺失,因此描述与实际行为存在实质性不匹配。
The code has shell/process-execution capability via subprocess, but that capability is not transparently declared despite affecting local execution and data handling. In skill context, hidden execution capability is more dangerous because the stated purpose is content improvement, not running external tools chosen by environment.
The code has shell/process-execution capability via subprocess, but that capability is not transparently declared despite affecting local execution and data handling. In skill context, hidden execution capability is more dangerous because the stated purpose is content improvement, not running external tools chosen by environment.
npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.
The cron-based auto flow describes automatic extraction and application of P0 rules to a target SKILL.md without a strong warning that a scheduled process may modify prompt/instruction files autonomously. In this context, automatic prompt mutation can cause integrity drift, unexpected behavior changes, or propagation of unsafe rules without timely human review.
The skill explicitly stores original drafts, final human-edited versions, and context in local logs, but does not prominently warn about privacy, retention, or sensitive-content handling. This can lead users to persist confidential business, personal, or regulated data in plaintext locations without informed consent or safeguards.
The skill defines persistent logging of full AI-generated drafts, human-finalized content, and context into local JSONL files. This creates a durable data collection channel that can accumulate sensitive prompts, proprietary text, secrets, and personal data, increasing exposure through local compromise, backups, syncing, or accidental sharing.
The skill instructs users to run npx clawhub@latest install self-improving-skill, which fetches and executes the latest package version at install time without pinning or integrity controls. That creates a supply-chain risk: a compromised upstream package or malicious new release could execute arbitrary code on the user's machine.
The module docstring and command/help text are written in Chinese, and the prompts sent to the LLM are also Chinese-only. This imposes a specific language/locale without opt-in or explanation, which matches the policy category for language or locale constraints.
A writing-style self-improvement skill mainly needs to read logs, analyze diffs, and update SKILL.md. Spawning external binaries—especially a user-specified command—introduces a broader execution capability that is not inherently required by that purpose as described in the manifest.
The script sends current SKILL.md content and human edit logs to external LLM CLIs without explicit notice, consent, redaction, or provider-boundary disclosure. Those logs may contain sensitive user text, internal guidance, or proprietary content, making silent exfiltration a significant privacy and compliance risk.
subprocess module calls execute external commands. Without careful input validation, this enables command injection.
]
for cmd in candidates:
try:
result = subprocess.run(cmd, input=prompt, capture_output=True,
text=True, timeout=timeout)
if result.returncode == 0 and result.stdout.strip():
return result.stdout.strip()
Data from a source is assigned to a variable that is later passed to a sink, creating a variable-mediated taint flow.
]
for cmd in candidates:
try:
result = subprocess.run(cmd, input=prompt, capture_output=True,
text=True, timeout=timeout)
if result.returncode == 0 and result.stdout.strip():
return result.stdout.strip()
The skill is presented as a writing-style improver, but it includes a configurable external command execution path that can run arbitrary binaries. This mismatch increases deception risk and can hide dangerous capability behind an innocuous description.
The command is taken from the IMPROVE_LLM_CMD environment variable and executed as a process, allowing an attacker who can influence the environment to run an arbitrary binary under the user's privileges. Although split() avoids shell metacharacter expansion, it still permits arbitrary program execution and unsafe argument parsing.
custom_cmd = os.environ.get("IMPROVE_LLM_CMD")
if custom_cmd:
try:
result = subprocess.run(custom_cmd.split(), input=prompt,
capture_output=True, text=True, timeout=timeout)
if result.returncode == 0 and result.stdout.strip():
return result.stdout.strip()
This sink combines untrusted prompt content with an arbitrarily configurable executable from IMPROVE_LLM_CMD, creating a genuine arbitrary-program-execution capability. While the prompt itself is stdin data, the environment-controlled command makes the overall flow dangerous in a multi-user or compromised execution environment.
custom_cmd = os.environ.get("IMPROVE_LLM_CMD")
if custom_cmd:
try:
result = subprocess.run(custom_cmd.split(), input=prompt,
capture_output=True, text=True, timeout=timeout)
if result.returncode == 0 and result.stdout.strip():
return result.stdout.strip()
The prompt includes up to 3000 characters each of original and final human-edited content plus contextual metadata and part of the current SKILL.md, all of which may be sensitive. Sending these materials to an external model can expose private user content and internal operational rules beyond what is necessary for rule extraction.
Data from a source is assigned to a variable that is later passed to a sink, creating a variable-mediated taint flow.
return None
name = f"SKILL-{datetime.now().strftime('%Y%m%d-%H%M%S')}.md"
backup_path = backup_dir / name
shutil.copy2(target, backup_path)
print(f"📦 备份: {backup_path}")
return backup_path
The merge step transmits the full proposal and the complete current SKILL.md to the model and asks it to regenerate the entire file, which can disclose sensitive policy content and amplify prompt injection risks. Regenerating the whole file also increases the blast radius of hallucinations or malicious inserted instructions.
The script writes untrusted LLM output directly into the target SKILL.md, enabling prompt-injection-driven persistence of malicious instructions or corruption of policy files. In auto mode this becomes unattended modification, increasing the chance of silent compromise of future agent behavior.
print("❌ 合并失败")
return
target.write_text(updated)
new_content = proposal_content.replace(
"status: pending",
Auto mode performs unattended extraction and application of P0 rules to SKILL.md, allowing persistent policy changes without human verification. Because the source material and merge logic are LLM-driven, prompt injection or misclassification can silently poison future outputs.
Data from a source is assigned to a variable that is later passed to a sink, creating a variable-mediated taint flow.
# 保存当前版本
if target.exists():
emergency = backup_dir / f"SKILL-pre-rollback-{datetime.now().strftime('%Y%m%d-%H%M%S')}.md"
shutil.copy2(target, emergency)
shutil.copy2(latest, target)
print(f"✅ 已回滚到: {latest.name}")
Data from a source is assigned to a variable that is later passed to a sink, creating a variable-mediated taint flow.
emergency = backup_dir / f"SKILL-pre-rollback-{datetime.now().strftime('%Y%m%d-%H%M%S')}.md"
shutil.copy2(target, emergency)
shutil.copy2(latest, target)
print(f"✅ 已回滚到: {latest.name}")
The module docstring, usage notes, help text, and console messages are written in Chinese only, which imposes a language choice on users without offering an alternative. This can violate language/locale policy where user-facing language should be selectable or explicitly justified.
The script writes the full original content to local JSONL logs, including potentially sensitive user text, with no minimization, consent prompt, retention control, or redaction. In a self-improving writing skill, inputs may contain drafts, proprietary material, credentials pasted by mistake, or personal data, so persistent plaintext storage materially increases confidentiality risk if the machine, backup system, or shared home directory is accessed.