Skillv0.1.11

ClawScan security

Eval Driven Development · ClawHub's context-aware review of the artifact, metadata, and declared behavior.

Scanner verdict

BenignMar 23, 2026, 2:14 AM

Verdict: benign
Confidence: high
Model: gpt-5-mini
Summary: The skill's instructions, small helper script, and requested actions are coherent with its stated purpose of adding/evaluating tests for Python LLM apps; it does not request unexplained credentials or perform suspicious remote installs.
Guidance: This skill appears to do what it says: it will read and edit your Python project, attempt to upgrade the 'pixie-qa' package, create a pixie_qa/ directory with datasets/tests, and run tests. Before installing or running it: (1) run it in a development environment or branch (not production), (2) back up or commit your repo so changes to files/lockfiles can be reviewed, (3) expect it to require network access for pip/poetry and possibly LLM API keys (it will stop and ask if keys like OPENAI_API_KEY are missing), and (4) review any proposed code edits before allowing iterative fixes — the skill's workflow requires explicit confirmation before making fixes beyond setup. The included version check fetches SKILL.md from raw.githubusercontent.com, which is standard for GitHub-hosted version checks.

Review Dimensions

Purpose & Capability: okName/description (eval-driven development for Python LLM apps) matches the actual behavior: reading code, instrumenting, building datasets, running tests, and optionally iterating. The included check_version helper and pixie API reference are relevant to the stated purpose.
Instruction Scope: noteSKILL.md instructs the agent to read and edit project files, run package manager commands (uv/poetry/pip), create a pixie_qa/ directory and run tests. This is broad (file edits and package upgrades) but appropriate for a tool that 'sets up evals' — the skill also documents hard gates (stop for missing API keys) and requires confirmation before applying fixes.
Install Mechanism: okThere is no install spec (instruction-only). The only code file (check_version.py) fetches a SKILL.md from raw.githubusercontent.com to compare versions — a reasonable version-checking behavior. No downloads from obscure or shortened URLs or archive extraction are present.
Credentials: okThe skill declares no required environment variables or credentials. The SKILL.md explicitly notes that certain evaluators need LLM API keys (e.g., OPENAI_API_KEY) and instructs the agent to stop and request them rather than guessing. There are no unrelated credential requests.
Persistence & Privilege: notealways:false and no automatic installation are appropriate. The skill will modify the user's project (create pixie_qa/, add tests, run installs) when invoked — this is expected but requires explicit user consent and care (it may change lockfiles or installed packages).