ClawHub
Back to skill

Security audit

Skill Benchmark

Security checks for vulnerabilities and agentic risk

Overview

This skill is an offline quality-scoring tool with an optional CI verifier; its code execution is purpose-aligned and disclosed, not hidden or destructive.

Safe to install for normal review use. Run the benchmark on ordinary local skill folders; run ci/verify_product.py only on repositories whose test code you are willing to execute. Prefer the reviewed package copy over the README's unpinned curl command if supply-chain reproducibility matters.

SkillSpector

Vulnerability Patterns
  • Output HandlingUnvalidated Output Injection, Cross-Context Output, Unbounded Output
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (9)

subprocess module call

Medium
Category
Dangerous Code Execution
Content
def has_self_test(fp):
    r = subprocess.run([sys.executable, fp, "self-test"], capture_output=True, text=True, timeout=30)
    return r.returncode == 0 and "PASS" in r.stdout
Confidence
70% confidence
Finding
subprocess module calls execute external commands. Without careful input validation, this enables command injection.

subprocess module call

Medium
Category
Dangerous Code Execution
Content
# 7 deploy-ready
    ci = os.path.join(os.path.dirname(os.path.abspath(__file__)), "ci_check.py")
    if os.path.isfile(ci) and os.path.isfile(skill):
        r = subprocess.run([sys.executable, ci], cwd=folder, capture_output=True, text=True)
        all_ok &= axis(r.returncode == 0, "deploy-ready (ci/ci_check.py PASS)")
    else:
        all_ok &= axis(True, "deploy-ready (skipped)")
Confidence
70% confidence
Finding
subprocess module calls execute external commands. Without careful input validation, this enables command injection.

subprocess module call

Medium
Category
Dangerous Code Execution
Content
all_ok &= axis(False, "frontmatter (no SKILL.md)")
        all_ok &= axis(False, "docs (no SKILL.md)")
    # 3 compiles
    comp_ok = all(subprocess.run([sys.executable, "-m", "py_compile", fp],
                                capture_output=True).returncode == 0 for fp in files)
    all_ok &= axis(comp_ok, "compiles (py_compile all .py)")
    # 4 self-test (any of 3 valid forms)
Confidence
70% confidence
Finding
subprocess module calls execute external commands. Without careful input validation, this enables command injection.

subprocess module call

Medium
Category
Dangerous Code Execution
Content
tests = [os.path.join(folder, f) for f in os.listdir(folder)
                 if (f.startswith("test_") or f.endswith("_test.py")) and f.endswith(".py")]
        if tests:
            st_ok = all(subprocess.run([sys.executable, t], capture_output=True, text=True,
                                      timeout=30).returncode == 0 for t in tests)
            st_note = f"{len(tests)} test_*.py"
        elif re.search(r"(?im)^test\s*:", skill_txt):
Confidence
70% confidence
Finding
subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Intent-Code Divergence

Medium
Confidence
91% confidence
Finding
The docstring presents the tool as a verification harness performing checks such as structure, docs, and security, which implies inspection-oriented behavior. In practice, the code also runs each discovered product script with a `self-test` argument and executes test files directly, introducing code execution side effects not reflected in the stated check descriptions.

Unvalidated Output Injection

High
Category
Output Handling
Content
def has_self_test(fp):
    r = subprocess.run([sys.executable, fp, "self-test"], capture_output=True, text=True, timeout=30)
    return r.returncode == 0 and "PASS" in r.stdout
Confidence
95% confidence
Finding
Model output is used without validation or sanitization. Unvalidated output injected into downstream contexts (SQL, shell, HTML) enables injection attacks and arbitrary code execution.

Unvalidated Output Injection

High
Category
Output Handling
Content
all_ok &= axis(False, "frontmatter (no SKILL.md)")
        all_ok &= axis(False, "docs (no SKILL.md)")
    # 3 compiles
    comp_ok = all(subprocess.run([sys.executable, "-m", "py_compile", fp],
                                capture_output=True).returncode == 0 for fp in files)
    all_ok &= axis(comp_ok, "compiles (py_compile all .py)")
    # 4 self-test (any of 3 valid forms)
Confidence
95% confidence
Finding
Model output is used without validation or sanitization. Unvalidated output injected into downstream contexts (SQL, shell, HTML) enables injection attacks and arbitrary code execution.

Unvalidated Output Injection

High
Category
Output Handling
Content
tests = [os.path.join(folder, f) for f in os.listdir(folder)
                 if (f.startswith("test_") or f.endswith("_test.py")) and f.endswith(".py")]
        if tests:
            st_ok = all(subprocess.run([sys.executable, t], capture_output=True, text=True,
                                      timeout=30).returncode == 0 for t in tests)
            st_note = f"{len(tests)} test_*.py"
        elif re.search(r"(?im)^test\s*:", skill_txt):
Confidence
95% confidence
Finding
Model output is used without validation or sanitization. Unvalidated output injected into downstream contexts (SQL, shell, HTML) enables injection attacks and arbitrary code execution.

Unvalidated Output Injection

High
Category
Output Handling
Content
# 7 deploy-ready
    ci = os.path.join(os.path.dirname(os.path.abspath(__file__)), "ci_check.py")
    if os.path.isfile(ci) and os.path.isfile(skill):
        r = subprocess.run([sys.executable, ci], cwd=folder, capture_output=True, text=True)
        all_ok &= axis(r.returncode == 0, "deploy-ready (ci/ci_check.py PASS)")
    else:
        all_ok &= axis(True, "deploy-ready (skipped)")
Confidence
95% confidence
Finding
Model output is used without validation or sanitization. Unvalidated output injected into downstream contexts (SQL, shell, HTML) enables injection attacks and arbitrary code execution.

Static analysis

No suspicious patterns detected.