ClawHub
Back to skill

Security audit

Improvement Evaluator

Security checks for vulnerabilities and agentic risk

Overview

This is a disclosed skill-evaluation tool, but real runs can send skill content, prompts, rubrics, and outputs through the local Claude CLI.

Install only if you are comfortable running local evaluator scripts and using the configured Claude CLI for non-mock evaluations. Use --mock or deterministic judges for sensitive or CI runs, keep secrets and proprietary data out of task prompts/rubrics/SKILL.md content, run only trusted task suites, and write state/cache outputs to a dedicated workspace.

SkillSpector

Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Findings (5)

subprocess module call

Medium
Category
Dangerous Code Execution
Content
f"Respond with ONLY a JSON object: {{\"score\": <float>, \"reasoning\": \"<str>\"}}"
        )
        try:
            result = subprocess.run(
                ["claude", "-p", "--output-format", "json"],
                input=prompt,
                capture_output=True,
Confidence
93% confidence
Finding
This subprocess sends rubric text and up to 3000 characters of AI output to an external LLM CLI, creating a data exfiltration boundary and allowing untrusted model output to influence a downstream model. In this skill context, evaluation data may include sensitive prompts, outputs, or proprietary test content, so invoking an external model without isolation or explicit disclosure is a real security and privacy concern.

Lp3

Medium
Category
MCP Least Privilege
Confidence
91% confidence
Finding
The skill advertises and instructs use of `python3 scripts/evaluate.py`, reads task suites and skill files, writes state/output artifacts, and can invoke external tooling such as `pytest` and potentially the `claude` CLI, which are meaningful file and shell capabilities. Having these capabilities without an explicit permissions declaration is dangerous because operators and orchestrators cannot accurately assess or constrain what the skill may access or execute, increasing the risk of unintended command execution, unsafe fixture/test behavior, or filesystem modification during evaluation.

Context-Inappropriate Capability

Medium
Confidence
90% confidence
Finding
The skill manifest describes pre-defined task-suite execution and pass/fail evaluation, but this code adds a general external-LLM judging path via subprocess. That expands the skill's execution and data-handling surface beyond a deterministic evaluator, increasing the risk of unintended networked model use, privacy issues, and prompt-injection-style influence from evaluated content.

Intent-Code Divergence

Medium
Confidence
93% confidence
Finding
The task says the rewrite must remove Tier 1 AI patterns, but the actual judge only checks for the presence of the token "is," which does not validate the stated requirement and can invert the intended behavior. This makes the evaluation harness unreliable: a weak or noncompliant model can pass, and a correct model can fail, allowing misleading execution_pass_rate results and undermining trust in the skill-improvement pipeline.

Missing User Warnings

Medium
Confidence
95% confidence
Finding
The code forwards rubric content and AI output to the claude CLI without any visible warning, consent, or disclosure mechanism. In an evaluation pipeline, those inputs can contain confidential prompts, benchmark data, or model outputs, so silent transmission to an external LLM is a meaningful privacy and compliance vulnerability.

Static analysis

No suspicious patterns detected.