SkillProbe
v1.0.0A/B evaluates any AI agent skill's real impact through three-role isolation (orchestrator + two sub-agents). Generates skill profiles, synthetic test tasks,...
⭐ 0· 163·1 current·1 all-time
MIT-0
Download zip
LicenseMIT-0 · Free to use, modify, and redistribute. No attribution required.
Security Scan
OpenClaw
Benign
high confidencePurpose & Capability
Name, description, SKILL.md, DISPATCH_PROTOCOL.md, SCORING_REFERENCE.md, and the helper script are coherent: all are focused on designing tasks, dispatching two isolated sub-agents, scoring, and reporting. No unrelated binaries, env vars, or config paths are requested.
Instruction Scope
Instructions stay within the evaluator role (profile target SKILL.md, generate tasks, dispatch two sub-agents, score). A key behavioral detail: Sub-Agent B receives the full skill content and both arms' prompts are sent to the configured LLM provider. That is expected for evaluation but means evaluated skill content (including anything embedded in its SKILL.md) will be transmitted to the LLM provider.
Install Mechanism
No install spec; instruction-only plus an optional local helper script. No downloads from external URLs or archive extraction. The helper script is benign and only attempts to invoke an existing runtime/CLI if present.
Credentials
The skill declares no required environment variables or credentials. The helper script's security manifest notes that a configured runtime/SkillProbe CLI may access provider environment variables at runtime — this is plausible for a tool that dispatches to an LLM provider, but users should be aware that a local CLI invocation could read environment-configured provider credentials.
Persistence & Privilege
No elevated privileges requested (always:false). The skill does not request persistent system-wide changes and does not modify other skills' configs. It is an orchestrator/workflow and does not demand permanent presence.
Assessment
This skill is internally coherent and does what it claims: it A/B evaluates other skills by sending tasks and the target skill's content to your configured LLM provider and scoring the outputs locally. Before running: (1) do not include secrets, API keys, or sensitive credentials in the SKILL.md or the skill bundle you evaluate — those will be sent to the LLM provider; (2) confirm which LLM provider/runtime is configured and whether you trust its data handling; (3) if using the local CLI helper, be aware it may read runtime/provider config or env vars needed by your LLM client; (4) run evaluations in a sandbox or with redacted skill content if you need to protect sensitive artifacts. If you want an extra safety check, inspect the specific skill bundle being evaluated and remove any embedded secrets before using SkillProbe.Like a lobster shell, security has layers — review code before you run it.
latestvk97drw0msv9zhfx5vyq5w1z8ds83a4kz
License
MIT-0
Free to use, modify, and redistribute. No attribution required.
Runtime requirements
🔬 Clawdis