Skill Regression

• Outcome: Review
• Latest audit: Jun 22, 2026 · 10:50 AM UTC
• Version: 1.0.3

Overview

This appears to be a legitimate regression-testing tool, but it needs Review because it can run shell commands from target test files and send skill contents/results to external LLM services.

Install only if you are comfortable running a regression framework with local command execution and external LLM calls. Use it on skills and TEST.md files you trust, prefer --skip-agent or local-only workflows for sensitive code, avoid enabling SR_REPORT_UPLOAD_HOOK unless the destination is trusted, and review or remove any OpenClaw probe cron job after testing.

SkillSpector

Vulnerability Patterns

• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Output Handling: Unvalidated Output Injection, Cross-Context Output, Unbounded Output
• Tool Misuse: Tool Parameter Abuse, Chaining Abuse, Unsafe Defaults
• Behavioral AST: exec() Call, eval() Call, Dynamic Import
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands

Findings (10)

subprocess module call

Medium

• Category: Dangerous Code Execution

Content

# meaning any command in TEST.md will execute. Only trust TEST.md from trusted sources.
        # ⚠️  Output merging: actual_output = stdout + stderr,
        # which is risky for exact-mode assertions (stderr warnings break match). Prefer contains/regex.
        proc = subprocess.run(
            script_cmd,
            shell=True,
            cwd=skill_dir,

• Confidence: 99% confidence
• Finding: The script command is taken from test case data and executed with shell=True, so any metacharacters, command chaining, or injected shell syntax in script_cmd will be interpreted by the shell. In this skill, the whole purpose is to run regression test commands from skill-controlled test definitions, which makes arbitrary command execution not incidental but built into the design, creating a real code execution risk if the tested skill or its test metadata is untrusted.

Missing User Warnings

Medium

• Confidence: 87% confidence
• Finding: The README documents a configurable upload hook that can exfiltrate generated regression reports to any destination, but it does not clearly warn that reports may contain sensitive skill contents, prompts, outputs, filesystem paths, or model responses. In a testing framework, reports often aggregate exactly the data users may not intend to publish, so normalizing upload without a privacy warning creates a real data-leak risk even if the feature is intentionally user-configured.

Missing User Warnings

Medium

• Confidence: 84% confidence
• Finding: The skill instructs users to provide an API key and send prompts/content to an external LLM endpoint but does not clearly warn that target skill contents, tests, outputs, and possibly sensitive data may leave the local system. In a regression-testing context, this is especially relevant because the analyzed skill or generated reports may contain proprietary code, credentials, or internal prompts.

Missing User Warnings

Medium

• Confidence: 89% confidence
• Finding: The optional upload hook allows generated reports to be sent off-system, but the documentation does not warn that reports may contain prompts, outputs, file paths, failure traces, or other sensitive material. Because hooks are arbitrary scripts, this also expands the exfiltration surface beyond the primary LLM backend.

Missing User Warnings

Medium

• Confidence: 87% confidence
• Finding: simulate_skill_response sends full SKILL.md content and the user trigger to an external OpenAI-compatible endpoint, which may expose proprietary prompts, embedded secrets, or sensitive user data to a third party. In a regression-testing skill, this is materially risky because the whole purpose is to transmit other skills' contents for analysis, increasing the chance of leaking confidential instructions or data without explicit consent or redaction.

Missing User Warnings

Medium

• Confidence: 88% confidence
• Finding: score_response sends the user input, expected behavior, and actual model response to an external LLM for grading, which can leak sensitive test cases, business logic, or user-provided content. Because this framework is designed for QA and auditing, those artifacts may contain confidential prompts, internal policies, or regulated data, making undisclosed third-party transmission a real privacy and compliance issue.

Missing User Warnings

Medium

• Confidence: 90% confidence
• Finding: infer_test_cases transmits slices of SKILL.md and README.md to an external LLM to generate tests, potentially exposing proprietary workflow details, hidden prompts, credentials accidentally stored in docs, or internal repository information. In this skill's context, that danger is elevated because it is specifically built to analyze arbitrary target skills, so it may routinely process sensitive third-party content without transparent disclosure.

Missing User Warnings

Medium

• Confidence: 94% confidence
• Finding: When TEST.md is absent, the script sends SKILL.md and README.md contents to an external OpenAI-compatible LLM to infer test cases. That can expose proprietary instructions, embedded secrets, internal URLs, or sensitive business logic without any explicit disclosure or consent at the point of use, which is a real data-leakage risk in a regression-testing tool.

Unvalidated Output Injection

High

• Category: Output Handling

Content

# meaning any command in TEST.md will execute. Only trust TEST.md from trusted sources.
        # ⚠️  Output merging: actual_output = stdout + stderr,
        # which is risky for exact-mode assertions (stderr warnings break match). Prefer contains/regex.
        proc = subprocess.run(
            script_cmd,
            shell=True,
            cwd=skill_dir,

• Confidence: 96% confidence
• Finding: Untrusted output-producing commands are executed directly from test case content, enabling an analyzed skill to smuggle attacker-controlled shell commands into the regression runner. Because this framework audits third-party skills, the context makes output/command injection especially dangerous: a malicious skill author can turn a 'test' into host-level command execution during analysis.

Tool Parameter Abuse

High

• Category: Tool Misuse

Content

# meaning any command in TEST.md will execute. Only trust TEST.md from trusted sources.
        # ⚠️  Output merging: actual_output = stdout + stderr,
        # which is risky for exact-mode assertions (stderr warnings break match). Prefer contains/regex.
        proc = subprocess.run(
            script_cmd,
            shell=True,
            cwd=skill_dir,

• Confidence: 98% confidence
• Finding: This is a classic tool-parameter-abuse issue: a powerful tool invocation (subprocess.run) is parameterized by externally supplied data and amplified by shell=True. The skill context increases severity because a regression-testing framework is expected to inspect potentially adversarial skills, yet it grants those skills a path to execute arbitrary local commands during testing.

Static analysis

No suspicious patterns detected.