Skillv1.0.0

ClawScan security

Flaky Test Detective · ClawHub's context-aware review of the artifact, metadata, and declared behavior.

Scanner verdict

BenignApr 29, 2026, 11:35 PM

Verdict: benign
Confidence: high
Model: gpt-5-mini
Summary: The skill's instructions and actions align with its stated purpose (finding and fixing flaky tests); it is instruction-only and coherent, but the SKILL.md references external CLIs/tools without listing them in the metadata and will run tests and access CI artifacts — review and authenticate tools before use.
Guidance: This skill appears to do what it says, but before using it: 1) ensure the command-line tools the SKILL.md uses are installed and authenticated (gh, npm/pytest/go, ripgrep, python3); the metadata doesn't list them so you'll need to provide them. 2) Be aware the skill will run your test suite repeatedly and may execute tests that perform network calls, modify databases, or change state — run it in an isolated environment (CI job, container, or a disposable branch) if you have side-effectful tests. 3) The GH CLI steps require access to repository CI runs/artifacts (gh auth/GH_TOKEN); confirm you are comfortable granting that access. 4) The skill writes temporary files (e.g., /tmp/run-*.txt) and may suggest code changes — review any automated fixes before applying. 5) If you want the skill to run automatically, consider limiting its permissions or requiring manual invocation. Overall it is coherent for its purpose but verify tooling and credentials and execute in a safe environment.

Review Dimensions

Purpose & Capability: noteThe skill's commands (gh, npm/pytest/go test, rg, python3) are appropriate for detecting flaky tests. However, the registry metadata declares no required binaries or credentials even though the runtime instructions clearly depend on tools like the GitHub CLI (gh), test runners, ripgrep (rg), and python3. This is an omission in declared requirements but not fundamentally incoherent with the stated purpose.
Instruction Scope: okSKILL.md stays on-topic: it reads CI run history, downloads test result artifacts, runs tests repeatedly, scans repository test files for patterns, and generates fixes/quarantine guidance. These actions are expected for flaky-test analysis. The instructions do instruct the agent to run local tests and read repository files and CI artifacts — normal for this purpose but potentially side-effectful.
Install Mechanism: okThere is no install spec and no code files (instruction-only), so nothing is written to disk by an installer. This lowers install-time risk.
Credentials: noteThe skill declares no required environment variables, but the GH CLI usage and artifact downloads will require GitHub authentication (gh auth / GH_TOKEN) and access to the repository. Test runs may also require credentials or service endpoints depending on the repo. The requested environment access is consistent with purpose but the metadata should have declared these dependencies so users know what credentials/permissions are needed.
Persistence & Privilege: okalways is false and the skill is user-invocable. It does not request persistent or elevated platform privileges. Autonomous invocation is enabled by default (normal), but there are no other persistence mechanisms or cross-skill config modifications.