RateMyClaw

Score your OpenClaw agent setup against similar agents. Scans your workspace, generates a local embedding for privacy-preserving semantic matching, and submi...

Security Scan

The skill is internally consistent with its stated purpose: it locally scans your OpenClaw workspace, maps files to a fixed taxonomy, generates a local embedding, and submits structured tags + an embedding to ratemyclaw.com (it does not transmit raw file contents).

Purpose & Capability

Name/description match the implementation: the code scans a workspace, maps signals to a fixed taxonomy, generates a local embedding (MiniLM or TF-IDF) and submits tags/embedding to ratemyclaw.com. The only credential referenced is an optional RATEMYCLAW_API_KEY which is appropriate for the stated remote API.

Instruction Scope

The runtime instructions and scripts perform a local workspace scan and may read selected files to detect tags, but the submit path only sends structured tags, skill slugs, maturity counts and the numeric embedding. The SKILL.md and code both assert that raw file contents and secrets are not transmitted. Minor documentation inconsistency: one place in SKILL.md lists '384 floats' as what gets sent (true for MiniLM) but TF-IDF produces a taxonomy-sized vector — the code handles either case and records the embedding_method. Review the generated_profile.json before submission as the skill instructs.

Install Mechanism

This is an instruction-only skill with included Python scripts and a small requirements.txt (scikit-learn). No opaque downloads or extracted archives are used. The scripts may invoke pip (or run pip via subprocess) to install scikit-learn and the user is prompted; sentence-transformers is suggested optionally (large model from PyPI/HuggingFace). The install behavior is proportionate to embedding generation.

Credentials

Only an optional RATEMYCLAW_API_KEY is used and documented. If absent, the script prompts before creating a free key via POST to the skill's documented API endpoint and saves it locally as .ratemyclaw_key with restrictive file permissions. No other unrelated credentials or environment variables are requested.

Persistence & Privilege

The skill is not always-enabled and does not request elevated platform privileges. It will create a local key file (.ratemyclaw_key) in the skill directory when you accept key generation — this is reasonable and is written with chmod 600. The skill does not attempt to modify other skills or system-wide configs.

Assessment

This skill appears to do what it says: it scans your workspace locally, produces structured tags, optionally generates a local embedding, and submits only tags/embedding/maturity counts to ratemyclaw.com. Before using it: (1) Inspect the generated_profile.json (the skill asks you to review tags) to confirm nothing sensitive was mis-tagged. (2) Be aware embeddings can be a sensitive fingerprint; only send them if you accept that risk. (3) The script may prompt to install scikit-learn (or you can manually run pip install -r requirements.txt); sentence-transformers is optional and large. (4) If you do not want any network calls, do not approve API key generation or submission. (5) Note a small doc mismatch: TF-IDF embeddings are taxonomy-sized (not always 384 floats) — this is an informational inconsistency only.

Score your OpenClaw agent and see how it compares to others working on similar problems.

What It Does

1. Scans your workspace (SOUL.md, MEMORY.md, skills, scripts, integrations, etc.)
2. Maps files to a fixed taxonomy of ~230 tags — no raw file content is extracted
3. Generates a local embedding for privacy-preserving matching (auto-detects best available method)
4. Submits only tags + embedding (float array) + maturity counts to ratemyclaw.com
5. Returns your score, grade, and a link to your full breakdown on the web

Prerequisites

After installing the skill, install Python dependencies:

pip install -r skills/ratemyclaw/requirements.txt

This installs scikit-learn (~30MB) for TF-IDF embeddings.

When running this skill for a user, always check and install requirements first:

pip install -r <skill_dir>/requirements.txt

The submit script auto-detects and uses the best available embedding method:

Priority	Library	Install Size	Quality	Command
1 (best)	sentence-transformers	~1.5GB	Semantic understanding	pip install sentence-transformers
2 (required)	scikit-learn	~30MB	Keyword/taxonomy matching	pip install -r requirements.txt

If sentence-transformers is detected, it's used automatically. Otherwise TF-IDF is the default. The script will suggest the upgrade path after each run.

Quick Start

When the user asks to "rate my claw", "score my agent", "check my setup", or similar:

Step 1: Scan the workspace

python3 scripts/profile_generator.py ~/.openclaw/workspace

This produces a generated_profile.json in the skill directory.

Step 2: Review the profile with the user

Show them what tags were detected and what skills were found. They can correct false positives before submission.

Step 3: Submit to RateMyClaw

python3 scripts/submit_profile.py generated_profile.json

If no RATEMYCLAW_API_KEY env var is set and no saved key exists, the script will ask for confirmation before generating a free key via POST /v1/keys. Pass --yes to skip the prompt in automated contexts.

The submit script will:

• Generate a 384-dim embedding locally using sentence-transformers
• Submit tags + embedding + maturity counts to ratemyclaw.com
• Print your score, grade, and a link to the full breakdown

Step 4: View results!

The full breakdown, insights, and recommendations are on the website at your score URL — not in the terminal.

What Gets Sent

Sent to ratemyclaw.com:

• Taxonomy tags (domains, tools, patterns, integrations) — structured labels only
• Skill slugs (names of installed skills)
• 384 floating-point numbers (the embedding vector)
• Maturity counts (number of memory files, scripts, etc.)
• Automation level and stage

Never sent:

• Raw file contents (SOUL.md, MEMORY.md, scripts, secrets, etc.)
• Workspace text of any kind

About embeddings: If an embedding library is installed, a numeric vector is generated locally from your tag data. MiniLM produces a 384-dim semantic embedding; TF-IDF produces a taxonomy-sized sparse vector. While embeddings encode semantic meaning and cannot be trivially reversed into text, they should be treated as potentially sensitive — they represent a condensed fingerprint of your agent's focus areas. If no library is installed, no embedding is sent and scoring relies on tag overlap alone.

Credentials

• RATEMYCLAW_API_KEY — optional env var. If not set, the script checks for a saved key in .ratemyclaw_key (inside the skill directory). If no key exists anywhere, it prompts before generating one.
• Keys are free and generated via POST /v1/keys on ratemyclaw.com
• Saved key file (.ratemyclaw_key) is created with chmod 600 and listed in .gitignore

Files

• scripts/profile_generator.py — Workspace scanner (runs locally, produces JSON)
• scripts/submit_profile.py — Embedding generation + API submission (prompts before any network calls if no key exists)
• references/taxonomy.json — The fixed tag taxonomy (233 tags)