RateMyClaw

• Outcome: Review
• Latest audit: Sep 16, 2026 · 4:56 AM UTC
• Version: 0.5.1

Overview

The skill mostly does what it says, but it collects and submits more local environment metadata than its privacy description clearly discloses.

Review the generated profile before submitting and be aware it may include installed skill names, integration inferences from secret filenames, secret counts, maturity metrics, and OpenClaw model configuration. Install dependencies in an isolated environment, avoid --yes unless you accept API key creation and package installation, and do not submit profiles from untrusted or edited JSON files.

A.I.G

Vulnerability Patterns

• Insecure Dependencies: Introduces malicious components through unsafe dependency sources
• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review

Findings (3)

other

Warning

• Location: scripts/profile_generator.py:310

Finding

Undisclosed Collection and Transmission of Global Model Configuration

Content

View full analysis

Vulnerability Details

File Location: scripts/profile_generator.py:310-340; scripts/submit_profile.py:249-288
Vulnerability Type: Undisclosed telemetry and privacy exposure
Risk Level: Medium

Vulnerable Code

python

# OpenClaw config is typically at ~/.openclaw/openclaw.json
config_path = ws.parent / "openclaw.json"
if not config_path.exists():
    # Try common alternative locations
    for alt in [ws / "openclaw.json", Path.home() / ".openclaw" / "openclaw.json"]:
        if alt.exists():
            config_path = alt
            break

if not config_path.exists():
    return models

try:
    with open(config_path) as f:
        config = json.load(f)

    # Extract default model
    agents = config.get("agents", {})
    defaults = agents.get("defaults", {})
    model_config = defaults.get("model", {})

    if isinstance(model_config, str):
        models["default_model"] = model_config
    elif isinstance(model_config, dict):
        models["default_model"] = model_config.get("primary")
        models["fallback_models"] = model_config.get("fallbacks", [])

    # Check agent list for heartbeat model
    agent_list = agents.get("list", [])
    for agent in agent_list:
        hb = agent.get("heartbeat", {})
        if isinstance(hb, dict) and hb.get("model"):
            models["heartbeat_model"] = hb["model"]
            break

The collected data is included in the remote submission:

python

payload = {
    "profile": {
        "domains": profile.get("domains", []),
        "tools": profile.get("tools", []),
        "patterns": profile.get("patterns", []),
        "integrations": profile.get("integrations", []),
        "skills_installed": profile.get("skills_installed", []),
        "automation_level": profile.get("automation_level", "manual"),
        "stage": profile.get("stage", "building"),
    },
    "embedding": embedding,
   
...[truncated 2364 chars]

Remediation

View remediation

Remediation Suggestions

1. Remove the models field from profile generation and remote submission unless it is strictly required for scoring.
2. Restrict scanning to the workspace path explicitly supplied by the user; do not automatically inspect global configuration outside that boundary.
3. If model telemetry is required, disclose every collected model field in SKILL.md and README.md, including its purpose and retention policy.
4. Require explicit opt-in consent before reading global configuration or transmitting model information.
5. Display the exact final outbound JSON payload before submission and provide a way to remove individual fields.
6. Apply an allowlist to model values and avoid collecting any neighboring configuration fields.

T09 · Insecure Skill Coding Practices

Warning

• Location: scripts/submit_profile.py:249

Finding

Outbound “Taxonomy Tags Only” Guarantee Is Not Enforced at the Submission Boundary

Content

View full analysis

Vulnerability Details

File Location: scripts/submit_profile.py:249-288
Vulnerability Type: Missing input and outbound-payload validation
Risk Level: Medium

Vulnerable Code

python

def submit(profile_path: str):
    api_key = get_api_key()

    with open(profile_path) as f:
        profile = json.load(f)

    print("🔐 Generating embedding locally...")
    embedding, embedding_method = generate_embedding(profile)

    # Build the submission payload
    payload = {
        "profile": {
            "domains": profile.get("domains", []),
            "tools": profile.get("tools", []),
            "patterns": profile.get("patterns", []),
            "integrations": profile.get("integrations", []),
            "skills_installed": profile.get("skills_installed", []),
            "automation_level": profile.get("automation_level", "manual"),
            "stage": profile.get("stage", "building"),
        },
        "embedding": embedding,  # None if no method available
        "embedding_method": embedding_method,  # "minilm", "tfidf", or "none"
        "maturity": profile.get("maturity", {}),
        "models": profile.get("models", {}),
    }

    print("📤 Submitting to ratemyclaw.com (tags + embedding only)...")
    data = json.dumps(payload).encode()
    req = urllib.request.Request(
        f"{API_BASE}/v1/profile",
        data=data,
        headers={
            "Authorization": f"Bearer {api_key}",
            "Content-Type": "application/json",
        },
        method="POST",
    )

    try:
        with urllib.request.urlopen(req) as resp:
            result = json.loads(resp.read())

Technical Analysis

The submission script accepts an arbitrary JSON file and copies its selected fields into an authenticated remote request without validating them against the bundled taxonomy or a strict schema.

In particular:

• Domain, tool, pattern, a ...[truncated 1878 chars]

Remediation

View remediation

Remediation Suggestions

1. Define and enforce a strict JSON schema before embedding generation or network access.
2. Validate domains, tools, patterns, and integrations against the corresponding sets in taxonomy.json.
3. Validate automation level and stage against their fixed taxonomy lists.
4. Restrict skill slugs to a conservative character pattern and maximum length.
5. Reconstruct maturity from an explicit allowlist of expected keys and validate each value's type and range.
6. Remove models, or reconstruct it from explicitly allowed scalar and list fields with strict limits.
7. Reject unknown keys, nested objects where scalars are expected, oversized arrays, oversized strings, and excessive total payload size.
8. Print a redacted but complete representation of the outgoing payload and request explicit confirmation immediately before network transmission.
9. Store generated profiles with restrictive permissions and warn when submitting a profile not produced by the current scan.

T08 · Insecure Dependencies

Note

• Location: scripts/submit_profile.py:203

Finding

Automatic Installation of Unpinned Third-Party Dependencies

Content

View full analysis

Vulnerability Details

File Location: requirements.txt:1; scripts/submit_profile.py:203-224
Vulnerability Type: Non-reproducible dependency installation
Risk Level: Low

Vulnerable Code

requirements.txt permits any current or future scikit-learn release at or above version 1.0:

text

scikit-learn>=1.0

The submission script can invoke pip automatically:

python

# Check for requirements.txt relative to this script
req_file = Path(__file__).parent.parent / "requirements.txt"
install_cmd = f"pip install -r {req_file}" if req_file.exists() else "pip install scikit-learn"

# Support non-interactive mode via --yes flag
if "--yes" in sys.argv:
    answer = "y"
else:
    answer = input("     Install now? (~30MB) [Y/n] ").strip().lower()

if answer in ("", "y", "yes"):
    import subprocess
    print("  📦 Installing dependencies...")
    if req_file.exists():
        cmd = [sys.executable, "-m", "pip", "install", "-r", str(req_file)]
    else:
        cmd = [sys.executable, "-m", "pip", "install", "scikit-learn"]
    result = subprocess.run(cmd, capture_output=True, text=True)
    if result.returncode == 0:
        print("  ✓ Dependencies installed!")

Technical Analysis

The Skill can execute pip during normal operation, including automatically when --yes is provided. The dependency declaration contains only a lower version bound and no hashes or lock file. Pip may therefore resolve a different scikit-learn release and different transitive dependencies over time.

Package installation executes package build and installation behavior with the same operating-system privileges as the agent process. Although the audited package name is legitimate and there is no evidence of intentional dependency confusion or typosquatting, mutable dependency resolution weakens reproducibility and expands the supply-chain trust boundary.

The documentation also recommends insta ...[truncated 1232 chars]

Remediation

View remediation

Remediation Suggestions

1. Pin reviewed direct and transitive dependency versions using a lock file.
2. Use package hashes with pip's hash-checking mode to verify downloaded artifacts.
3. Test and publish separate lock files for supported Python versions and platforms where necessary.
4. Do not perform package installation automatically in --yes mode.
5. Require explicit user approval that displays the package index, exact resolved versions, and command to be executed.
6. Prefer installation into an isolated virtual environment rather than the agent's global Python environment.
7. Pin and hash optional sentence-transformers dependencies if that embedding path is supported.
8. Document trusted package sources and advise against unreviewed custom indexes.

SkillSpector

Vulnerability Patterns

• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access
• Supply Chain: Unpinned Dependencies, External Script Fetching, Obfuscated Code
• Excessive Agency: Unrestricted Tool Access, Autonomous Decision Making, Scope Creep
• Tool Misuse: Tool Parameter Abuse, Chaining Abuse, Unsafe Defaults
• Rogue Agent: Self-Modification, Session Persistence

Findings (16)

Tool Parameter Abuse

High

• Category: Tool Misuse
• Confidence: 80% confidence

Finding

Tool parameters are crafted to achieve unintended or unsafe behavior. Parameter abuse can bypass intended safety checks (e.g. shell=True, --force, dangerous glob patterns).

Content

Scanner excerpt · README.md (reported line 39)May include surrounding context.

md

- The 384-dim float array **can't be reversed** into text
- We never see your files, only structured tags and numbers
- Individual profiles are never exposed to other users — only aggregate cluster patterns
- Delete your profile anytime: `DELETE /v1/profile/{id}`

## Scoring

Tp4

High

• Category: MCP Tool Poisoning
• Confidence: 97% confidence

Finding

The skill description suggests a straightforward scoring workflow, but the documented/observed behavior includes creating and storing API keys, possible interactive pip installation, and sending additional profile attributes beyond tags and embeddings. Hidden credential creation, local persistence, package installation, and expanded telemetry materially change the security posture and can surprise users into granting network, execution, and persistence actions they did not knowingly approve.

Content

No source excerpt is available for this finding.

Tp4

High

• Category: MCP Tool Poisoning
• Confidence: 96% confidence

Finding

The skill description suggests a straightforward scoring workflow, but the documented/observed behavior includes creating and storing API keys, possible interactive pip installation, and sending additional profile attributes beyond tags and embeddings. Hidden credential creation, local persistence, package installation, and expanded telemetry materially change the security posture and can surprise users into granting network, execution, and persistence actions they did not knowingly approve.

Content

No source excerpt is available for this finding.

Credential Access

High

• Category: Privilege Escalation
• Confidence: 70% confidence

Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · scripts/submit_profile.py (reported line 30)May include surrounding context.

python

def get_api_key() -> str:
    """Get API key from env var or saved file. Prompts before generating a new one."""
    # 1. Check env var
    env_key = os.environ.get("RATEMYCLAW_API_KEY", "")
    if env_key.startswith("rmc_"):

Context-Inappropriate Capability

High

• Category: Not specified by scanner
• Confidence: 97% confidence

Finding

Executing pip via subprocess is a powerful capability not necessary for submitting a profile and meaningfully expands the attack surface. Installing packages can execute arbitrary setup/build/install hooks from transitive dependencies, making this risky in a skill that otherwise appears to perform local analysis and API submission.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium

• Category: MCP Least Privilege
• Confidence: 90% confidence

Finding

The skill advertises shell, filesystem, environment, and network-capable behavior but does not declare any explicit tool scope such as permissions or allowed-tools. That omission weakens reviewability and can let a broadly invoked skill perform sensitive workspace scanning, local file access, dependency installation, and outbound submission without a clear least-privilege contract.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium

• Category: Not specified by scanner
• Confidence: 91% confidence

Finding

Broad trigger phrases like 'rate my claw', 'score my agent', or 'check my setup' lack clear scope boundaries for a skill that scans the workspace and may perform network submission. Ambiguous invocation increases the risk of accidental activation in contexts where the user did not intend a full local scan or data submission.

Content

No source excerpt is available for this finding.

Sudo/Root Execution

Medium

• Category: Privilege Escalation
• Confidence: 80% confidence

Finding

Commands invoke sudo or root privileges. Verify this elevated access is necessary and justified.

Content

Scanner excerpt · SKILL.md (reported line 104)May include surrounding context.

md

- **`RATEMYCLAW_API_KEY`** — optional env var. If not set, the script checks for a saved key in `.ratemyclaw_key` (inside the skill directory). If no key exists anywhere, it prompts before generating one.
- Keys are free and generated via `POST /v1/keys` on ratemyclaw.com
- Saved key file (`.ratemyclaw_key`) is created with `chmod 600` and listed in `.gitignore`

## Files

Session Persistence

Medium

• Category: Rogue Agent
• Confidence: 60% confidence

Finding

Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.

Content

Scanner excerpt · scripts/profile_generator.py (reported line 90)May include surrounding context.

python

"rust": [".rs", "cargo"],
    "swift": [".swift", "xcode"],
    "bash": [".sh", "#!/bin/bash", "bash"],
    "sql": [".sql", "SELECT ", "CREATE TABLE"],
    "sqlite": ["sqlite", ".db"],
    "postgres": ["postgres", "psql", "pg_"],
    "redis": ["redis"],

Description-Behavior Mismatch

Medium

• Category: Not specified by scanner
• Confidence: 95% confidence

Finding

The code enumerates files in the .secrets directory, records secret filenames, and reports the number of configured secrets in the profile. Even without reading secret contents, secret names and counts can disclose which services, accounts, or infrastructure the user relies on, contradicting the claim that only fixed-taxonomy tags leave the machine. Because this skill is meant to profile an agent for remote comparison, such metadata disclosure is especially sensitive.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium

• Category: Not specified by scanner
• Confidence: 91% confidence

Finding

The script reads OpenClaw model configuration from local config files and includes model identifiers in the generated profile and console output, even though the skill description says it scans the workspace and submits tags plus an embedding. Model choices can reveal provider usage, operational setup, and agent architecture details that users may not expect to disclose. In this skill context, that metadata is adjacent to the data intended for remote scoring, so scope expansion makes the privacy risk more significant.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 88% confidence

Finding

The script writes the generated workspace profile to generated_profile.json on disk automatically after scanning, without an explicit warning, prompt, or ephemeral mode. That creates a persistent artifact containing derived metadata about the user's workspace, integrations, maturity, skills, and possibly model configuration, which can later be accessed by other local tools or accidentally shared. In a profiling skill, silent persistence increases privacy exposure beyond the immediate scan.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium

• Category: Not specified by scanner
• Confidence: 95% confidence

Finding

The skill modifies the host environment by installing Python packages, but this behavior is not aligned with or disclosed by the stated purpose of scanning, embedding, and submitting a profile. Undisclosed environment modification increases supply-chain and integrity risk because users may not expect code execution that persists beyond the session.

Content

No source excerpt is available for this finding.

subprocess module call

Medium

• Category: Dangerous Code Execution
• Confidence: 92% confidence

Finding

The script can invoke pip in a subprocess to install dependencies at runtime, which is executable behavior beyond simple profile submission. Even though the command is hardcoded and not shell-injected, it changes the local environment and may fetch and execute untrusted package code from package indexes or dependency chains.

Content

Scanner excerpt · scripts/submit_profile.py (reported line 223)May include surrounding context.

python

cmd = [sys.executable, "-m", "pip", "install", "-r", str(req_file)]
            else:
                cmd = [sys.executable, "-m", "pip", "install", "scikit-learn"]
            result = subprocess.run(cmd, capture_output=True, text=True)
            if result.returncode == 0:
                print("  ✓ Dependencies installed!")
                print()

Unpinned Dependencies

Low

• Category: Supply Chain
• Confidence: 95% confidence

Finding

The dependency is specified as scikit-learn>=1.0 without an upper bound or exact pin, which makes builds non-reproducible and can unintentionally pull in vulnerable or breaking releases over time. In a skill that performs local embedding generation, this is not directly exploitable by itself, but it increases supply-chain risk and makes it harder to verify whether a deployed environment is affected by known issues.

Content

Scanner excerpt · requirements.txt (reported line 1)May include surrounding context.

text

scikit-learn>=1.0

Unverifiable Dependency: scikit-learn has 6 known advisory(ies) (CVE-2020-13092 (scikit-learn Deserialization of Untrusted Data); CVE-2024-5206 (scikit-learn sensitive data leakage vulnerability); CVE-2020-28975 (scikit-learn Denial of Service) +3 more), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low

• Category: Supply Chain
• Confidence: 89% confidence

Finding

Because scikit-learn is unpinned, the manifest does not establish whether installation will resolve to a version affected by one of several known advisories. The skill's context—workspace scanning and embedding generation—could process potentially sensitive local data, so using an unverifiable dependency version modestly increases the risk of exposure or instability if a vulnerable release is installed.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.