Back to skill

Security audit

Improvement Executor

Security checks for vulnerabilities and agentic risk

Overview

This skill has a coherent file-editing purpose, but its scripts can modify or restore arbitrary local files without enough path checks and its safety claims are overstated.

Review before installing. Only use this skill in a constrained workspace with trusted ranking, receipt, and backup artifacts, and avoid running rollback with --execute unless the source backup and destination target are verified manually. Do not rely on the documented execute dry-run preview or rollback hash verification until the implementation actually enforces them.

Vulnerability Patterns
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (3)

T05 · Unauthorized Access and Privilege Escalation

Error
Location
scripts/execute.py:274
Finding

Candidate-Controlled Target Path Allows File Modification Outside the Approved Scope

Content
View full analysis

Vulnerability Details

File Location: scripts/execute.py:274-303
Vulnerability Type: Missing path authorization and scope validation
Risk Level: High

Vulnerable Code

python
target_file = Path(candidate["target_path"]).expanduser().resolve()
if not target_file.exists() or not target_file.is_file():
    result = {
        "status": "error",
        "modified": False,
        "reason": f"target file not found: {target_file}",
    }
    output_path = Path(args.output).expanduser().resolve() if args.output else paths["executions"] / f"{run_id}-{candidate['id']}.json"
    artifact = {
        "schema_version": SCHEMA_VERSION,
        "lane": ranking_artifact.get("lane", "generic-skill"),
        "run_id": run_id,
        "stage": "executed",
        "status": "error",
        "created_at": utc_now_iso(),
        "candidate_id": candidate["id"],
        "target_path": str(target_file),
        "result": result,
        "truth_anchor": str(output_path),
    }
    write_json(output_path, artifact)
    update_state(
        state_root, run_id=run_id, stage="executed", status="error",
        target_path=str(target_file), truth_anchor=str(output_path),
        extra={"candidate_id": candidate["id"]},
    )
    print(str(output_path))
    return 1

backup_path = paths["executions"] / "backups" / run_id / f"{candidate['id']}-{target_file.name}"
backup_file(target_file, backup_path)

plan = candidate.get("execution_plan", {})
action = plan.get("action")
handler = ACTION_HANDLERS.get(action)
if handler is not None:
    result = handler(target_file, plan)

Technical Analysis

The file that will be modified is derived directly from the candidate-controlled target_path. Calling resolve() canonicalizes the path but does not authorize it. The implementation does not verify that the resolved path:

  • Matches the trusted ranking_artifact["target"]["path"].

...[truncated 2004 chars]

Remediation
View remediation

Remediation Suggestions

  1. Derive the destination from a trusted project root rather than accepting an unrestricted absolute candidate path.

  2. Require the candidate target to exactly match the target declared by the trusted ranking artifact, where the workflow expects a single target.

  3. Canonicalize both the approved root and candidate path, then enforce containment:

    python
    approved_root = Path(configured_project_root).resolve(strict=True)
    target_file = Path(candidate["target_path"]).expanduser().resolve(strict=True)
    
    try:
        target_file.relative_to(approved_root)
    except ValueError:
        raise ValueError("Candidate target is outside the approved project root")
    
  4. Reject symbolic links and verify the target using descriptor-based, no-follow filesystem operations where supported.

  5. Validate ranking artifacts against a strict schema and authenticate artifacts when they cross a trust boundary.

  6. Apply an explicit extension or file allowlist appropriate to the selected action.

  7. Store backups in a directory with restrictive permissions and avoid backing up unauthorized targets.

  8. Add negative tests covering absolute paths, .. traversal, symlinks, mismatches with the ranking target, and paths outside the approved root.

T05 · Unauthorized Access and Privilege Escalation

Error
Location
scripts/rollback.py:74
Finding

Untrusted Rollback Pointers Allow Arbitrary Local File Overwrite

Content
View full analysis

Vulnerability Details

File Location: scripts/rollback.py:74-142 and scripts/rollback.py:145-177
Vulnerability Type: Unvalidated source and destination paths in file restoration
Risk Level: High

Vulnerable Code

Receipt-based rollback accepts an execution artifact path and then trusts both paths in its rollback pointer:

python
execution_path = receipt.get("truth_anchor", {}).get("execution_path")
if not execution_path:
    # 尝试从 receipt 中推断
    run_id = receipt.get("run_id")
    candidate_id = receipt.get("candidate_id")
    if run_id and candidate_id:
        executions_dir = DEFAULT_ROOT / "executions"
        for exec_file in executions_dir.glob(f"{run_id}-{candidate_id}.json"):
            execution_path = str(exec_file)
            break

if not execution_path:
    return {
        "status": "error",
        "reason": "Cannot find execution artifact",
        "receipt_path": str(receipt_path)
    }

execution = read_json(Path(execution_path))

# 获取 rollback pointer from result
result = execution.get("result", {})
rollback_pointer = result.get("rollback_pointer", {})
backup_path = rollback_pointer.get("backup_path")

if not backup_path:
    return {
        "status": "error",
        "reason": "No backup information found in execution artifact",
        "execution_path": execution_path
    }

# 获取目标路径 (truth_anchor is a string path, not a dict)
target_path = rollback_pointer.get("target_path")
if not target_path:
    return {
        "status": "error",
        "reason": "No target path found in execution artifact",
        "execution_path": execution_path
    }

# 执行回滚
backup_file = Path(backup_path)
target_file = Path(target_path)

if not backup_file.exists():
    return {
        "status": "error",
        "reason": f"Backup file not found: {backup_path}",
        "backup_path": backup_path
    }

result = {
    "status": "dry_
...[truncated 4190 chars]
Remediation
View remediation

Remediation Suggestions

  1. Remove unrestricted direct restoration or require an explicit trusted policy that authorizes the source and destination.
  2. Accept only execution artifacts generated under the configured state root.
  3. Authenticate receipts and execution artifacts, for example with a keyed message authentication code or signature managed outside attacker-controlled files.
  4. Bind the rollback destination to the original approved target recorded by the executor and reject any mismatch.
  5. Require backup sources to reside under the executor's dedicated backup directory.
  6. Resolve paths strictly and enforce containment with Path.relative_to() against trusted roots.
  7. Reject symlink sources and destinations and use no-follow, descriptor-based operations where available.
  8. Confirm that the source and destination are regular files and that the destination's identity matches the recorded original target.
  9. Restore through a temporary file in the destination directory, validate it, set restrictive permissions, and atomically replace the target.
  10. Add tests for forged receipts, external execution artifacts, traversal, symlinks, unauthorized destinations, and direct-mode misuse.

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/rollback.py:115
Finding

Rollback Restores Backups Without the Documented Integrity Verification

Content
View full analysis

Vulnerability Details

File Location: scripts/execute.py:320-325, scripts/rollback.py:115-142; documented behavior at SKILL.md:108-116
Vulnerability Type: Missing backup integrity verification
Risk Level: Medium

Vulnerable Code

The executor records paths but does not record an integrity digest:

python
"backup_path": str(backup_path),
"rollback_pointer": {
    "method": "restore_backup_file",
    "backup_path": str(backup_path),
    "target_path": str(target_file),
},

The rollback implementation verifies only that the backup exists before restoring it:

python
backup_file = Path(backup_path)
target_file = Path(target_path)

if not backup_file.exists():
    return {
        "status": "error",
        "reason": f"Backup file not found: {backup_path}",
        "backup_path": backup_path
    }

result = {
    "status": "dry_run" if dry_run else "success",
    "action": "rollback",
    "target_path": str(target_file),
    "backup_path": str(backup_file),
    "rollback_pointer": rollback_pointer,
    "receipt_path": str(receipt_path),
    "dry_run": dry_run
}

if not dry_run:
    # 创建目标文件的备份(回滚前的状态)
    pre_rollback_backup = target_file.parent / f"{target_file.name}.pre-rollback-{datetime.now().strftime('%Y%m%d-%H%M%S')}"
    if target_file.exists():
        shutil.copy2(target_file, pre_rollback_backup)
        result["pre_rollback_backup"] = str(pre_rollback_backup)

    # 恢复 backup
    shutil.copy2(backup_file, target_file)
    result["restored_from"] = str(backup_file)

The Skill documentation states that rollback verifies the backup's SHA-256 hash, but no hash is stored in the rollback pointer and no digest is calculated before restoration.

Technical Analysis

A backup is trusted solely based on its path and existence. If the backup is modified after creation, the altered content is silently restored. This violates the documented ...[truncated 1700 chars]

Remediation
View remediation

Remediation Suggestions

  1. Calculate SHA-256 immediately after creating each backup.
  2. Store the digest in the rollback pointer together with the canonical target path, backup path, creation time, and original file metadata.
  3. Recalculate and compare the digest immediately before restoration, using hmac.compare_digest() for comparison.
  4. Fail closed and do not modify the target if the digest is missing or mismatched.
  5. Authenticate the execution artifact or receipt with an HMAC or digital signature if attackers may modify both backups and metadata.
  6. Restrict backup directory and file permissions to the service account.
  7. Open and verify the same file descriptor that will be used for restoration to reduce time-of-check/time-of-use races.
  8. Record and verify target identity and expected path before replacement.
  9. Add tests that modify a backup after execution and assert that rollback rejects it without changing the target.
  10. Update the implementation and documentation together so the stated integrity guarantees match actual behavior.
Vulnerability Patterns
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (13)

Tp4

High
Category
MCP Tool Poisoning
Confidence
95% confidence
Finding

代码的核心确实是执行已批准候选对目标文件的修改,并在修改前备份,这与声明的大方向基本一致,也确实不承担打分或 gate 验证职责。但声明中两项重要能力与实际不符:第一,‘回滚’仅体现在输出 artifact 里的 rollback_pointer 和备份文件,当前脚本没有任何 restore backup 的执行路径;第二,‘预览变更效果’未实现,因为各 handler 在成功路径都会直接 write_text 写回文件,没有 preview/dry-run 开关。另一个次要不一致是声明中的 4 个 action 是抽象名,而实现要求的是更具体的 action key。综合看,描述夸大了能力边界,因此应判定为不匹配。

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

声明描述的是一个通用变更执行器,既能应用改进候选,也能回滚,还能预览,并明确支持四类编辑动作。实际代码却只是 rollback.py:它解析命令行参数后,仅执行两类回滚路径(从 receipt 推导 backup/target,或直接从 backup 恢复到 target),以及通过 run-id/candidate-id 定位 receipt。代码中没有任何 append、replace、insert_before、update_yaml 的编辑逻辑,也没有将“已批准的改进候选”写入目标文件的实现。dry-run 只是在回滚时模拟、不执行文件复制,不等同于声明中的变更预览能力。因此,描述与实际行为存在实质性不匹配。

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
95% confidence
Finding

代码片段是测试文件,不是完整实现,但从可见行为来看,其核心能力集中在对 Markdown 文档段落和 YAML frontmatter 的定向修改,以及生成 execution trace。声明描述的是一个更通用的“改进执行器”,包括应用候选、回滚、预览和自动备份。测试中确实能看到与声明部分对应的 4 类编辑操作的近似映射,但它们的实际名称和语义更窄:append/replace/insert 仅针对 Markdown section,update_yaml 仅针对 frontmatter。更重要的是,声明中的关键能力——回滚、预览、自动备份——在该代码片段中完全没有体现。依据题目标准,这属于描述与实际行为存在实质差异。

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
93% confidence
Finding

The skill describes file-reading and file-writing behavior but does not declare any explicit tool scope such as permissions or allowed-tools. For a skill that modifies target files and manages backups, missing capability boundaries increases the chance of unintended file access or misuse by the orchestrator or model at runtime.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

The trigger phrase "回滚" is overly generic for a skill that can restore files and interact with backup artifacts. Broad triggers increase the likelihood of accidental invocation on unrelated rollback requests, potentially causing unintended file restoration or modification in an automated pipeline.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The manifest says the skill can preview change effects, but each supported handler writes directly to the target file once it decides to modify content. There is no dry-run or preview flag in argument parsing or execution flow that would generate a diff without applying the change, so the stated preview function is not present in the actual code.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The manifest says the skill supports exactly four actions named append, replace, insert_before, and update_yaml. However, the code dispatches on different action identifiers: append_markdown_section, replace_markdown_section, insert_before_section, and update_yaml_frontmatter. This is a semantic mismatch between the advertised interface and the actual behavior clients must invoke.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The manifest explicitly says the skill should be used to roll back previous changes, but this executor only creates backups and emits a rollback_pointer in the output artifact. There is no code path, argument, or handler in this file that actually restores a backup or executes a rollback action, so the advertised rollback capability is not implemented here.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
99% confidence
Finding

The epilog says '默认 dry-run 模式,需要实际执行请去掉 --dry-run 标志', implying execution occurs simply by not passing --dry-run. In reality, --dry-run is hard-defaulted to True at L255-L260, and only --execute at L262-L272 switches it to real execution. This is a direct contradiction that can mislead operators about whether a rollback will actually happen.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
77% confidence
Finding

The natural-language description is presented primarily in Chinese while many operational terms remain in English, but the file does not state whether the skill should adapt to the user's preferred language. Because policy checks apply to all file types, this can be a locale/language policy concern if the skill defaults to one language without explicit choice.

Content

No source excerpt is available for this finding.

Overly Broad Trigger

Low
Category
Trigger Abuse
Confidence
85% confidence
Finding

A one-word trigger like "回滚" is highly collision-prone and can match many ordinary user requests. In a file-modifying skill, accidental activation can lead to unintended rollback flows, especially in orchestrated environments where trigger matching may be automatic.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Low
Category
Not specified by scanner
Confidence
84% confidence
Finding

The file docstring says this script is an "Executor for low-risk generic-skill candidates," while the skill manifest describes an executor specifically for approved improvement candidates with a defined action set. This is not just incomplete documentation; it describes a different intent and operational context than the skill claims.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Low
Category
Not specified by scanner
Confidence
96% confidence
Finding

The inline CLI documentation says rollback will create backups named like pre-rollback-*.bak. However, the implementation at L140 and L183 constructs filenames as <original>.pre-rollback-<timestamp> without a .bak suffix. This is an active documentation/code contradiction about the artifact the tool produces.

Content

No source excerpt is available for this finding.