skill-function-test

• Outcome: Review
• Latest audit: Jun 25, 2026 · 10:37 AM UTC
• Version: 1.16.1

Overview

This skill is a testing suite, but it can execute and modify target skills and includes under-disclosed ways to generate misleading reports.

Install only in a disposable or sandboxed workspace and only for targets you are willing to execute and modify. Treat reports from this skill as non-authoritative unless you remove or ignore the hook-bypass/fabricated-report script and verify tests ran from real logs.

SkillSpector

Vulnerability Patterns

• Excessive Agency: Unrestricted Tool Access, Autonomous Decision Making, Scope Creep
• Trigger Abuse: Overly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
• Behavioral AST: exec() Call, eval() Call, Dynamic Import
• MCP Least Privilege: Underdeclared Capability, Wildcard Permission, Missing Permission Declaration
• MCP Tool Poisoning: Hidden Instructions, Unicode Deception, Parameter Description Injection

Findings (53)

subprocess module call

Medium

• Category: Dangerous Code Execution

Content

capture_output=True, timeout=10)
    # 实际标记以上面为准，这里只是为了后续 end 能引用
    try:
        result = subprocess.run(
            cmd_args, capture_output=True, text=True, timeout=30,
        )
        wall = subprocess.run([sys.executable, "-c", "import time; print(time.perf_counter())"],

• Confidence: 94% confidence
• Finding: This function executes arbitrary cmd_args provided to it, with no validation, allowlisting, or trust boundary enforcement. In a security-testing engine that consumes blueprint- and plan-derived data, this can lead to execution of attacker-controlled commands or unintended local programs if upstream inputs are compromised.

subprocess module call

Medium

• Category: Dangerous Code Execution

Content

_tl(self.skill_dir, f"target_{sid}", f"{sid}: {label_detail}", "--type", "subprocess_wall")

        try:
            result = subprocess.run(
                cmd, capture_output=True, text=True, timeout=30,
                cwd=self.skill_dir,
            )

• Confidence: 98% confidence
• Finding: The engine executes target skill scripts discovered from blueprint metadata and manual plans, meaning untrusted skill code is run on the host. Since this is done directly with the current user's privileges and only a timeout boundary, a malicious skill can perform arbitrary filesystem, network, or process actions during testing.

subprocess module call

Medium

• Category: Dangerous Code Execution

Content

os.path.dirname(os.path.abspath(__file__)), "hooks.py"
))
def _hook_check(skill_dir, step):
    r = subprocess.run([sys.executable, _HOOKS_SCRIPT, "check", skill_dir, step],
                        capture_output=True, text=True, encoding="utf-8")
    if r.stdout and r.stdout.strip(): print(r.stdout)
    if r.stderr and r.stderr.strip(): print(r.stderr, file=sys.stderr)

• Confidence: 94% confidence
• Finding: The code invokes a secondary Python script with attacker-influenced arguments (`skill_dir`, `step`) and then trusts and relays its output. In this skill, hooks are part of a control plane and a compromised or malicious target skill directory can influence hook behavior or terminal output, making this more dangerous than a normal local helper invocation.

subprocess module call

Medium

• Category: Dangerous Code Execution

Content

for script, script_rel in cli_scripts:
            _tl(self.skill_dir, "d1_subprocess", f"D1: {script_rel} --help", "--type", "subprocess_wall")
            try:
                result = subprocess.run(
                    [sys.executable, script, "--help"],
                    capture_output=True, text=True, timeout=30,
                    cwd=self.skill_dir

• Confidence: 99% confidence
• Finding: The engine executes discovered project Python files with `--help` purely based on source-text detection of a `__main__` guard. In a security-analysis/test skill context, that means analyzing an untrusted skill can trigger arbitrary attacker-controlled code execution during testing, including filesystem, network, or environment access.

subprocess module call

Medium

• Category: Dangerous Code Execution

Content

print(f"  [TIMELINE] [START] [{phase}] {label}  @{rel}  ({start_mid})")

    try:
        result = _sp.run(
            cmd_args, capture_output=True, text=True, timeout=timeout, cwd=cwd
        )
    except _sp.TimeoutExpired:

• Confidence: 88% confidence
• Finding: `run_with_timing` executes arbitrary `cmd_args` supplied by callers with only a timeout and optional cwd, creating a generic command-execution sink inside the skill framework. In this skill context, which orchestrates testing of other skills and scripts, that makes it easier for untrusted or poorly validated workflow inputs to trigger execution of attacker-chosen local programs or scripts.

Lp3

Medium

• Category: MCP Least Privilege
• Confidence: 95% confidence
• Finding: The skill declares no permissions while its documented tooling and analyzer evidence indicate capabilities for environment access, file read/write, shell execution, and network use. This creates a dangerous transparency gap: a caller or policy layer may treat the skill as low-risk even though it can modify files, execute commands, and potentially exfiltrate data or fetch remote content.

Tp4

High

• Category: MCP Tool Poisoning
• Confidence: 97% confidence
• Finding: The documented behavior omits higher-risk functions such as bypass-style report generation, structural auto-repair/file creation, and starting a local HTTP server with browser launch. Hidden or under-documented capabilities are dangerous because they defeat user expectation, weaken review controls, and can be abused to fabricate outputs or modify the target environment beyond the stated testing workflow.

Intent-Code Divergence

High

• Confidence: 98% confidence
• Finding: The permissions document contains materially conflicting statements: earlier sections disclose 38 subprocess calls and 5 file deletions with MEDIUM/HIGH risk, while later sections claim there are no high-privilege operations and overall risk is LOW. This kind of contradiction can mislead reviewers and users into granting unified execution approval under false assumptions, reducing scrutiny over command execution and deletion capabilities.

Intent-Code Divergence

Medium

• Confidence: 95% confidence
• Finding: The file claims writes are limited to `data/output/`, but other sections document deletion activity in installation-directory scripts and describe output isolation under a different `.standardization/.../outputs/` path. Inconsistent scope descriptions undermine the trust boundary for file operations and may cause operators to approve a skill believing filesystem effects are narrower than they really are.

Intent-Code Divergence

High

• Confidence: 99% confidence
• Finding: The script explicitly advertises that it bypasses hooks and then proceeds to generate and write reports without running the declared gating workflow. In the context of a testing skill that claims hooks are mandatory blockers, this creates an integrity-bypass mechanism that can falsify compliance or test outcomes and conceal real failures.

Description-Behavior Mismatch

High

• Confidence: 99% confidence
• Finding: The script hardcodes fabricated test result data and feeds it directly into report generators, producing PASS-heavy outputs without executing the described tests. This undermines the trust model of the skill, allowing an operator to forge evidence of successful validation and potentially ship unsafe or noncompliant skills.

Context-Inappropriate Capability

Medium

• Confidence: 96% confidence
• Finding: A general-purpose hook-bypass capability is dangerous here because the skill's purpose is to validate workflows, and bypassing the controls defeats the very safeguards meant to prevent invalid outputs. In this context, the capability is more dangerous, not less, because it directly subverts the mandatory blocking controls advertised by the skill metadata.

Context-Inappropriate Capability

Medium

• Confidence: 97% confidence
• Finding: The generated HTML pulls JavaScript from external CDNs at report-view time, creating a supply-chain and privacy risk: opening the local report can trigger outbound network requests and execute third-party code in the browser. In a security-testing/reporting skill, this is more concerning because reports may be opened in sensitive environments and are expected to be self-contained.

Description-Behavior Mismatch

Medium

• Confidence: 84% confidence
• Finding: The scanner performs hidden side effects beyond passive analysis by invoking hook scripts that can block workflow execution and emit output. In a skill-inspection context, that expands trust boundaries and creates an unexpected execution path, especially because analyzed content is treated as potentially adversarial and the tool is not purely read-only.

Context-Inappropriate Capability

Medium

• Confidence: 86% confidence
• Finding: A blueprint scanner should primarily inspect files, but this script can execute external helper processes for blocking and marker recording. That extra capability increases attack surface and operational risk because compromise or misuse of those helpers affects more than the scan itself.

Description-Behavior Mismatch

Medium

• Confidence: 87% confidence
• Finding: The script writes artifacts into a centralized directory outside the target skill tree, which is a non-obvious side effect for a component described mainly as a scanner/output generator. Cross-directory writes can overwrite or leak data, surprise operators, and make containment harder when analyzing untrusted skills.

Intent-Code Divergence

High

• Confidence: 99% confidence
• Finding: stage_7_fix 的 docstring 写明“LLM 过滤误报 + 自动修复”，manifest 也描述包含“修复”阶段。但实现仅汇总失败项、读取源码上下文并打印“[FP]/[FIX]”判断提示，然后直接记录日志返回，没有任何实际代码修改、修复调用或补丁应用逻辑。

Description-Behavior Mismatch

Medium

• Confidence: 82% confidence
• Finding: The bump stage may update the version whenever fix_mode is enabled and issues exist, even if no real remediation occurred. In a testing and release orchestration skill, this can falsely signal that corrective changes were made, undermining auditability and potentially causing unreviewed or insecure artifacts to be treated as remediated releases.

Description-Behavior Mismatch

High

• Confidence: 98% confidence
• Finding: The module-level documentation says S4 'only reports, does not repair', but the file contains repair logic that creates and modifies files under the target skill directory. This mismatch is dangerous because operators may run the tool assuming it is read-only, while it can alter project contents and potentially mask integrity issues or modify sensitive files through crafted paths.

Intent-Code Divergence

High

• Confidence: 97% confidence
• Finding: The code claims to perform sequential noise injection and evaluate held/failed behavior, but in reality it auto-generates trace entries marking every result as '坚守'. This can produce false assurance in a security or fidelity test pipeline, causing downstream reports to misrepresent system robustness and potentially allowing unsafe skills to pass review.

Vague Triggers

Medium

• Confidence: 86% confidence
• Finding: Broad natural-language triggers can cause the skill to activate for loosely related requests, which is risky because this skill performs backups, writes reports, changes configuration, and may initiate repair workflows. In context, unintended invocation is more dangerous than usual because the skill is explicitly designed to take filesystem-modifying actions automatically and discourages asking the user for confirmation.

Vague Triggers

Medium

• Confidence: 88% confidence
• Finding: The manifest trigger field contains many aliases with broad operational scope, increasing the chance that the skill is selected when the user intended a lighter-weight review or a different testing mode. Because the skill includes shell/file/network-adjacent capabilities and persistent writes, over-triggering can lead to unanticipated side effects on local project state.

Missing User Warnings

Medium

• Confidence: 90% confidence
• Finding: The skill instructs writing reports, configuration, and conclusion files to disk, while also emphasizing that the agent should not ask the user configuration questions and should proceed automatically. That combination is dangerous because it normalizes persistent modification of the workspace without a clear user-facing warning about what will be created or altered.

Vague Triggers

Medium

• Confidence: 93% confidence
• Finding: The trigger list is extremely broad and bundles many loosely related phrases, which can cause the skill to activate in contexts the user did not intend. Because this skill performs backups, testing, repair, version bumping, and report writing, accidental invocation could launch disruptive workflow steps on the wrong target or at the wrong time.

Natural-Language Policy Violations

Medium

• Confidence: 89% confidence
• Finding: The changelog explicitly states a policy of '配置即方案，禁止询问用户' and directs the agent to read `.test-config.json` and execute without asking the user. In an agent skill, this weakens human-in-the-loop safeguards and can cause the agent to perform sensitive or destructive actions based solely on repository-controlled configuration, which is untrusted input under the threat model.

Static analysis

Detected: suspicious.dynamic_code_execution

Dynamic code execution detected.

Critical

• Code: suspicious.dynamic_code_execution
• Location: scripts/test_engine.py:433