Skillv1.0.1

ClawScan security

Llm As Judge · ClawHub's context-aware review of the artifact, metadata, and declared behavior.

Scanner verdict

SuspiciousFeb 26, 2026, 10:32 PM

Verdict: suspicious
Confidence: medium
Model: gpt-5-mini
Summary: The skill's description matches an LLM-evaluation ensemble, but its declared runtime requirements (both cloud API keys required and a local ollama binary) are stricter than the documentation implies and may be unnecessary; this incoherence warrants caution.
Guidance: This skill appears to implement what it says — a three-layer judge ensemble that uses local validators plus sampled cloud LLM judges — but there are a couple of practical mismatches to consider before installing: - It declares both ANTHROPIC_API_KEY and OPENAI_API_KEY as required. If you only intend to run a single cloud judge or only local Ollama inference, you should confirm whether both keys are actually needed or if the skill could accept one optionally. Avoid supplying unnecessary credentials. - It lists ollama as a required binary even though local use is described as optional. If you won't run Ollama locally, ensure the skill can operate without it rather than installing/using a local runtime you don't want. - Because the skill performs outbound network calls to third-party APIs, expect billing and data sent to Anthropic/OpenAI. Review the upstream behavior (repository or code) to confirm what request payloads include and whether any sample data or logs would be transmitted. If you decide to use it: provide only the credentials needed for your intended mode (prefer least privilege), monitor API usage/costs during initial runs, and review the upstream repo (https://github.com/reddinft/skill-llm-as-judge) or any implementation before giving full access to production credentials.

Review Dimensions

Purpose & Capability: noteThe skill claims to build an ensemble that uses local (Ollama) and cloud judges (Anthropic + OpenAI). Requiring python3 plus ollama, and both ANTHROPIC_API_KEY and OPENAI_API_KEY is consistent with the described two-cloud-plus-local architecture. However, the docs state judges are sampled and local inference can be used alone; making ollama and both cloud keys mandatory (in requires lists) is stricter than the description implies and may be unnecessary for some legitimate uses (e.g., cloud-only or local-only evaluation).
Instruction Scope: okSKILL.md contains architecture, validation and scoring rules, and example Python usage. It explicitly describes network outbound calls to Anthropic and OpenAI and local inference via Ollama. The instructions do not direct the agent to read unrelated system files or other secrets, nor do they instruct exfiltration to unexpected endpoints.
Install Mechanism: okThis is instruction-only (no install spec and no code files), so nothing is written to disk by an installer. That lowers installation risk. The skill does rely on external binaries being present on PATH rather than installing them itself.
Credentials: concernThe skill requires two cloud API keys (ANTHROPIC_API_KEY and OPENAI_API_KEY) and marks Anthropic as the primary credential. Requiring both keys is reasonable if the ensemble always needs two cloud judges, but the doc describes sampling and optional components (e.g., optional Gemini tiebreaker, local Ollama use). Making both cloud keys mandatory and insisting on an ollama executable is disproportionate to some described modes of operation and reduces flexibility. If you only plan to run local/offline evaluations or only one cloud provider, forcing both keys and ollama may be unnecessary and increases secret exposure.
Persistence & Privilege: okThe skill is not always-enabled and does not request special persistent privileges. It does permit autonomous invocation by default (platform normal), but there's no indication it modifies other skills or system-wide configs.