ClawHub
Back to skill

Security audit

Skill Benchmark

Security checks for vulnerabilities and agentic risk

Overview

The core benchmark script is limited, but the package tells users to run unpinned remote code and includes a verifier that can execute candidate project code during checks.

Before installing, prefer the bundled skill_benchmark.py over the curl command, or only use a pinned release with a verified hash. Do not run ci/verify_product.py on untrusted skill folders on a normal workstation or credentialed CI runner; if dynamic tests are needed, run them in a disposable, secret-free sandbox with restricted filesystem and network access.

A.I.G

Vulnerability Patterns
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (3)

T03 · Remote Payload Retrieval and Execution

Warning
Location
README.md:36
Finding

Mutable Remote Python Payload Is Downloaded Without Integrity Verification

Content
View full analysis
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Error
Location
ci/verify_product.py:34
Finding

Product Verification Harness Executes Python Files from Untrusted Target Folders

Content
View full analysis
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Note
Location
ci/ci_check.py:14
Finding

CI Checker Silently Skips Validation Because It Resolves the Package Root Incorrectly

Content
View full analysis
Remediation
View remediation

SkillSpector

Vulnerability Patterns
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Findings (6)

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The script advertises a security check for hardcoded secrets, but it also executes untrusted product code as part of self-test verification. In CI/security tooling context, this contradiction is dangerous because a malicious repository can pass superficial checks while using the verifier itself as an execution path to steal CI secrets or tamper with results.

Content

No source excerpt is available for this finding.

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
96% confidence
Finding

This code executes an arbitrary product Python file as a subprocess to probe for a self-test mode. Because fp points to untrusted repository content being verified, any top-level code or hostile self-test implementation will run with the CI runner's privileges, turning verification into code execution.

Content

Scanner excerpt · ci/verify_product.py (reported line 37)May include surrounding context.

python
def has_self_test(fp):
    r = subprocess.run([sys.executable, fp, "self-test"], capture_output=True, text=True, timeout=30)
    return r.returncode == 0 and "PASS" in r.stdout

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · ci/verify_product.py (reported line 67)May include surrounding context.

python
all_ok &= axis(False, "frontmatter (no SKILL.md)")
        all_ok &= axis(False, "docs (no SKILL.md)")
    # 3 compiles
    comp_ok = all(subprocess.run([sys.executable, "-m", "py_compile", fp],
                                capture_output=True).returncode == 0 for fp in files)
    all_ok &= axis(comp_ok, "compiles (py_compile all .py)")
    # 4 self-test (any of 3 valid forms)

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
97% confidence
Finding

This executes every discovered test_*.py or *_test.py file directly from the untrusted product directory. Any attacker can place malicious code in a test file and gain code execution in CI or on a developer machine during verification, potentially exfiltrating secrets or modifying the environment.

Content

Scanner excerpt · ci/verify_product.py (reported line 79)May include surrounding context.

python
tests = [os.path.join(folder, f) for f in os.listdir(folder)
                 if (f.startswith("test_") or f.endswith("_test.py")) and f.endswith(".py")]
        if tests:
            st_ok = all(subprocess.run([sys.executable, t], capture_output=True, text=True,
                                      timeout=30).returncode == 0 for t in tests)
            st_note = f"{len(tests)} test_*.py"
        elif re.search(r"(?im)^test\s*:", skill_txt):

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · ci/verify_product.py (reported line 92)May include surrounding context.

python
# 7 deploy-ready
    ci = os.path.join(os.path.dirname(os.path.abspath(__file__)), "ci_check.py")
    if os.path.isfile(ci) and os.path.isfile(skill):
        r = subprocess.run([sys.executable, ci], cwd=folder, capture_output=True, text=True)
        all_ok &= axis(r.returncode == 0, "deploy-ready (ci/ci_check.py PASS)")
    else:
        all_ok &= axis(True, "deploy-ready (skipped)")

Intent-Code Divergence

Low
Category
Not specified by scanner
Confidence
86% confidence
Finding

The FAQ states "Offline? A: Yes," which communicates an offline-oriented intent for the tool. However, the installation section instructs users to download the script via curl from GitHub, which contradicts that claim in the documentation context, even if the runtime tool itself may operate locally afterward.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.