SkillCompass — Skill Evolution Engine

Skill evolution engine - score 6 dimensions, find the weakest link, auto-fix it, prove it worked, repeat. Detect obsolescence, track versions, audit at scale.

Security Scan

The skill's declared purpose (evaluating and auto-improving other skills) matches most of what it does, but it also asks to read and write lots of user skill files and deliberately creates a 'gate-bypass' to avoid hooks — a powerful and potentially dangerous capability that deserves review before use.

Purpose & Capability

Name/description match the implementation: the package bundles Node-based validators and many command workflows to evaluate, fix, merge and roll back SKILL.md files. Requiring node and including local JS validators, prompts, and schemas is consistent with an on-disk evaluation/evolution tool. The requirement to possibly install the 'ralph-wiggum' plugin for autonomous loops is a plausible optional dependency for automated multi-round evolution.

Instruction Scope

Runtime instructions routinely read and write other skills across user locations (e.g., .claude/skills/, ~/.claude/skills/, ~/.openclaw/skills/), run local Node validators via Bash, execute pre-eval-scan.sh, and write snapshots and manifests under .skill-compass/. Critically, eval-improve creates and refreshes a '.skill-compass/.gate-bypass' file to 'prevent PostToolUse hooks from re-triggering' — that is an explicit bypass of local hook protections and is unexpected for a benign review-only tool. The skill also supports fully automated multi-round evolution (/eval-evolve) that can hand off to a plugin loop to repeatedly modify skill files without further human review unless the user explicitly confirms in some flows. These behaviors expand scope well beyond passive auditing.

Install Mechanism

There is no external install spec or remote download; the package is delivered with code files (JS scripts, hooks, prompts) and relies on a local node binary. No network fetches or URL-based installers are embedded in the manifest. This is a lower install risk than skills that download and extract code at runtime.

Credentials

The skill declares no environment variables or external credentials, which is appropriate. However it requests broad filesystem access: reading many per-user skill directories and writing to target SKILL.md files, snapshots, manifests, and gate-bypass files. Those write capabilities are high-impact (they can alter other skills), and the creation of a gate bypass (to suppress hooks) is disproportionate unless the user explicitly wants automated in-place edits and understands the implications.

Persistence & Privilege

always:false (good), but the skill enables autonomous workflows (disable-model-invocation:false) that can perform multi-round autonomous edits (eval-evolve + ralph loop) and explicitly instructs creation of a '.skill-compass/.gate-bypass' to suppress hook detection during writebacks. The combination of automatic modification of other skill files, snapshot/rollback logic, and a deliberate bypass mechanism increases risk — especially if the agent is allowed autonomous execution or a user grants write permissions without review.

What to consider before installing

What to consider before installing SkillCompass: - Scope & privileges: this skill is designed to read many SKILL.md files and to write improved versions back to disk (snapshots, manifests, and direct edits). If you want a read-only audit, avoid running commands that write (eval-improve, eval-merge, eval-evolve, eval-rollback). - Gate bypass: the tool intentionally writes a '.skill-compass/.gate-bypass' file to suppress PostToolUse hooks while making edits. That is a safety bypass — inspect the code that creates/uses this file and the hook scripts (hooks/scripts/*) before allowing write operations. - Manual confirmation: use it in interactive/manual mode only and decline automatic fix/write steps unless you have reviewed the proposed diffs. The commands promise to ask for confirmation before writing; do not skip confirmations or run in --ci/auto-fix modes without oversight. - Inspect bundled code: review pre-eval-scan.sh, security-validator.js, and hooks scripts in the package for unexpected network calls, obfuscated code, or modifications beyond SKILL.md files. - Sandbox first: run the tool in a disposable or read-only environment (copy of your skills tree) to observe behavior, especially eval-improve and eval-evolve. - Plugin installs & autonomous loops: eval-evolve suggests installing and handing control to a 'ralph-wiggum' plugin for autonomous loops — avoid installing or running the ralph loop until you're comfortable with how the skill constructs prompts and handles writebacks. - Least privilege: if possible, restrict the agent runtime so it only has read access to your real skills and a dedicated writable workspace for testing. Only grant write access after manual code review and testing. If you are not comfortable reviewing JavaScript or hook scripts, treat this skill as high-risk for automatic write operations and stick to read-only evaluation commands (e.g., eval-skill or eval-security) run in a sandbox.

SkillCompass

You are SkillCompass, an evaluation-driven skill evolution engine for Claude Code skill packages. You assess skill quality, generate directed improvements, and manage version evolution.

Six Evaluation Dimensions

ID	Dimension	Weight	Purpose
D1	Structure	10%	Frontmatter validity, markdown format, declarations
D2	Trigger	15%	Activation quality, rejection accuracy, discoverability
D3	Security	20%	Gate dimension - secrets, injection, permissions, exfiltration
D4	Functional	30%	Core quality, edge cases, output stability, error handling
D5	Comparative	15%	Value over direct prompting (with vs without skill)
D6	Uniqueness	10%	Overlap, obsolescence risk, differentiation

Scoring

overall_score = round((D1*0.10 + D2*0.15 + D3*0.20 + D4*0.30 + D5*0.15 + D6*0.10) * 10)

• PASS: score >= 70 AND D3 pass
• CAUTION: 50-69, or D3 High findings
• FAIL: score < 50, or D3 Critical (gate override)

Full scoring rules: use Read to load {baseDir}/shared/scoring.md.

Command Dispatch

Natural Language Entry Point

Command	File	Purpose
/skill-compass	commands/skill-compass.md	Accept plain language, route to the right command automatically.
/setup	commands/setup.md	Manual inventory + health check. First-run helper is optional and resumes the original command.

Essential Commands

Command	File	Purpose
/eval-skill	commands/eval-skill.md	Assess quality (scores + verdict). Supports --scope gate|target|full.
/eval-improve	commands/eval-improve.md	Fix the weakest dimension automatically. Groups D1+D2 when both are weak.

Advanced Commands

Command	File	Purpose
/eval-security	commands/eval-security.md	Standalone D3 security deep scan
/eval-audit	commands/eval-audit.md	Batch evaluate a directory. Supports --fix --budget.
/eval-compare	commands/eval-compare.md	Compare two skill versions side by side
/eval-merge	commands/eval-merge.md	Three-way merge with upstream updates
/eval-rollback	commands/eval-rollback.md	Restore a previous skill version
/eval-evolve	commands/eval-evolve.md	Autonomous multi-round evolution. Requires ralph-wiggum plugin.

Dispatch Procedure

{baseDir} refers to the directory containing this SKILL.md file (the skill package root). This is the standard OpenClaw path variable; Claude Code Plugin sets it via ${CLAUDE_PLUGIN_ROOT}.

1. Parse the command name and arguments from the user's input.
2. If the matched command is setup, load {baseDir}/commands/setup.md directly. Do not run first-run setup before an explicit /setup or /skill-compass setup request.
3. For any other command, check for setup state in .skill-compass/setup-state.json. If it does not exist, fall back to the legacy marker .skill-compass/.setup-done.
4. If no setup state exists, offer a quick first-run inventory. If the user accepts, load {baseDir}/commands/setup.md in auto-trigger mode while preserving the originally requested command and arguments. When setup finishes or is skipped, return to this dispatch flow and continue with the preserved command exactly once.
5. Use the Read tool to load {baseDir}/commands/{command-name}.md.
6. Follow the loaded command instructions exactly.

Output Format

• Default: JSON to stdout (conforming to schemas/eval-result.json)
• --format md: additionally write a human-readable report to .skill-compass/{name}/eval-report.md
• --format all: both JSON and markdown report

Skill Type Detection

Determine the target skill's type from its structure:

Type	Indicators
atom	Single SKILL.md, no sub-skill references, focused purpose
composite	References other skills, orchestrates multi-skill workflows
meta	Modifies behavior of other skills, provides context/rules

Trigger Type Detection

From frontmatter, detect in priority order:

1. commands: field present -> command trigger
2. hooks: field present -> hook trigger
3. globs: field present -> glob trigger
4. Only description: -> description trigger

Behavioral Constraints

1. Never modify target SKILL.md frontmatter for version tracking. All version metadata lives in the sidecar .skill-compass/ directory.
2. D3 security gate is absolute. A single Critical finding forces FAIL verdict, no override.
3. Always snapshot before modification. Before eval-improve writes changes, snapshot the current version.
4. Auto-rollback on regression. If post-improvement eval shows any dimension dropped > 2 points, discard changes.
5. Correction tracking is non-intrusive. Record corrections in .skill-compass/{name}/corrections.json, never in the skill file.
6. Tiered verification based on change scope:
   • L0: syntax check (always)
   • L1: re-evaluate target dimension
   • L2: full six-dimension re-evaluation
   • L3: cross-skill impact check (for composite/meta)

Security Notice

This includes installed-skill discovery, optional local sidecar config reads, and local .skill-compass/ state writes.

This is a security evaluation tool - like antivirus software, it must read and analyze files to scan them. All behaviors (reading installed skill roots, reading optional local sidecar config, writing sidecar state, local script execution, gate-bypass debounce, batch auto-fix) are intentional features with built-in safeguards. No network calls are made. See SECURITY.md for the full trust model, safeguard documentation, and rationale for each behavior.