Back to skill

Security audit

Ranking Of Claws

Security checks for vulnerabilities and agentic risk

Overview

This leaderboard skill is mostly aligned with its stated purpose, but it installs ongoing reporting and transmits session-derived usage metadata with some important under-disclosed behavior.

Review carefully before installing. This skill will report OpenClaw usage metadata to an external leaderboard and can keep doing so through cron or hooks. Confirm the identity in config.json, remove or regenerate the bundled Pika configuration, and only enable cron/hooks if you are comfortable with recurring uploads of token, model, and possible cost metadata.

Vulnerability Patterns
  • System PersistenceInstalls backdoors, hooks, services, or scheduled tasks that survive the run
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (4)

T06 · System Persistence

Error
Location
scripts/setup-cron.sh:6
Finding
Automatic Installation of Persistent Scheduled Telemetry Reporting<![CDATA[ ## Vulnerability Details **File Location**: `scripts/setup-cron.sh:6-20`; invoked automatically by `scripts/install.sh:22-25` and `scripts/install.sh:51` **Vulnerability Type**: Persistent scheduled task **Risk Level**: High ### Vulnerable Code ```bash LOG_FILE="${HOME}/.openclaw/ranking-of-claws-cron.log" MARKER="# ranking-of-claws" CRON_LINE="*/10 * * * * bash \"$REPORT_SCRIPT\" >> \"$LOG_FILE\" 2>&1 $MARKER" CURRENT_CRON="$(crontab -l 2>/dev/null || true)" if printf '%s\n' "$CURRENT_CRON" | grep -Fq "$MARKER"; then echo "ranking-of-claws: cron already configured." exit 0 fi { printf '%s\n' "$CURRENT_CRON" printf '%s\n' "$CRON_LINE" } | sed '/^[[:space:]]*$/N;/^\n$/D' | crontab - ``` The installer invokes this code automatically: ```bash if [ "$FORCE" != "1" ] && [ -n "${EXISTING_NAME:-}" ]; then echo "ranking-of-claws: already registered as \"$EXISTING_NAME\" (config.json kept)." echo "To re-register: ROC_FORCE_REREGISTER=1 bash scripts/install.sh" bash "$SCRIPT_DIR/setup-cron.sh" exit 0 fi ``` ```bash bash "$SCRIPT_DIR/setup-cron.sh" ``` ### Technical Analysis The installation process modifies the current user's crontab and registers `report.sh` to run every ten minutes. The scheduled task survives the installer process, shell termination, Agent sessions, and system reboots where cron is enabled. The recurring script reads OpenClaw session files from `~/.openclaw/agents/*/sessions/*.jsonl`, derives token and model usage, and sends the resulting deltas to an external API. Although automatic cron installation is disclosed in `SKILL.md`, persistent execution is not the minimum privilege required for manual leaderboard reporting. No separate consent prompt or opt-in control is presented before modifying the crontab. The script preserves existing crontab content and uses a fixed marker to avoid duplicate registration. However, the project does not provide an uninstall procedure to remove the marked entry. Deleting the Skill ...[truncated 1154 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Make scheduled reporting explicitly opt-in rather than installing it as an unconditional part of registration. 2. Display the exact schedule, executable path, log path, data sources, and destination before requesting consent. 3. Separate registration from persistence setup, for example: ```bash bash scripts/install.sh bash scripts/setup-cron.sh --enable ``` 4. Provide a dedicated uninstall operation that removes only the line containing the exact `# ranking-of-claws` marker. 5. Tie recurring execution to the Skill's enabled lifecycle through a platform-managed scheduler where possible. 6. Verify that the report script and its parent directories are owned by the expected user and are not group- or world-writable before registering the job. 7. Use an absolute path to a validated shell or invoke the executable script directly. 8. Document how to inspect, disable, and remove the scheduled task. 9. Avoid installing both cron reporting and hook-based periodic reporting, because redundant mechanisms broaden the monitoring and persistence surface. ]]>

T09 · Insecure Skill Coding Practices

Warning
Location
config.json:1
Finding
Bundled Registration Identity Silently Bypasses User Registration<![CDATA[ ## Vulnerability Details **File Location**: `config.json:1-6`; `scripts/install.sh:20-25` **Vulnerability Type**: Insecure pre-populated configuration **Risk Level**: Medium ### Vulnerable Code The distributed configuration contains an existing identity: ```json { "agent_name": "Pika", "country": "CH", "gateway_id": "01dce40cbe5ad053", "registered_at": "2026-03-05T07:40:55Z" } ``` The installer trusts any existing Agent name and skips registration: ```bash EXISTING_NAME="$(get_existing_agent_name || true)" if [ "$FORCE" != "1" ] && [ -n "${EXISTING_NAME:-}" ]; then echo "ranking-of-claws: already registered as \"$EXISTING_NAME\" (config.json kept)." echo "To re-register: ROC_FORCE_REREGISTER=1 bash scripts/install.sh" bash "$SCRIPT_DIR/setup-cron.sh" exit 0 fi ``` ### Technical Analysis `SKILL.md` states that installation prompts for an Agent name and writes a local registration. However, the package already contains a populated `config.json`. Because `install.sh` treats any nonempty `agent_name` as evidence of prior local registration, a normal installation preserves the bundled `Pika` identity and immediately proceeds to cron setup. The `gateway_id` is also pre-generated rather than being derived uniquely during installation. As a result, independent users can transmit telemetry under the same Agent and gateway identity. This is not a credential disclosure because the identifier is not shown to authenticate privileged API access. It is nevertheless unsafe configuration handling that enables telemetry misattribution and undermines the claimed registration process. ### Attack Path 1. A user installs the package without changing `config.json`. 2. `install.sh` reads the bundled `agent_name` value. 3. The installer concludes that the user is already registered. 4. The Agent-name prompt and local gateway-ID generation are skipped. 5. The installer enables recurring reporting. 6. The user's usage is submitted under the bundled `P ...[truncated 638 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Do not distribute a generated registration identity. Ship an empty template such as: ```json { "agent_name": "", "country": "XX", "gateway_id": "", "registered_at": "" } ``` 2. Prefer creating a separate runtime configuration outside the installed package. 3. Generate `gateway_id` locally on first registration. 4. Only treat an existing configuration as valid if all required fields are present and it was explicitly created by the local installation process. 5. Prompt the user to confirm an existing identity before preserving it. 6. Consider adding a configuration-version field and an installation-specific marker. 7. Ensure package publication workflows remove generated local identity data before release. 8. Add an installation test asserting that a fresh package prompts for registration and does not contain a pre-generated gateway identifier. ]]>

other

Warning
Location
hooks/handler.js:128
Finding
Hook Sends Undisclosed Cost Telemetry to an External Service<![CDATA[ ## Vulnerability Details **File Location**: `hooks/handler.js:128-139` and `hooks/handler.js:309-323`; inconsistent with `SKILL.md:91-95` **Vulnerability Type**: Undisclosed telemetry collection **Risk Level**: Medium ### Vulnerable Code The hook extracts cost information from session usage records: ```javascript let cost = 0; if (usage.cost && typeof usage.cost === "object") { cost = safeNumber(usage.cost.total); } else { cost = safeNumber(usage.cost); } if (total <= 0 && input <= 0 && output <= 0 && cost <= 0) return null; return { total, input, output, cost }; ``` It then includes the cost delta in outbound reports: ```javascript const ok = await postReport({ gateway_id: config.gateway_id, agent_name: config.agent_name, country: config.country || "XX", tokens_delta: delta.tokens > 0 ? delta.tokens : delta.input + delta.output, tokens_in_delta: delta.input, tokens_out_delta: delta.output, cost_delta: delta.cost, model, }); ``` The privacy statement is narrower than the actual hook payload: ```markdown ## Privacy - Only agent name, country, and token counts are shared - No message content transmitted - Gateway ID is a non-reversible hash ``` ### Technical Analysis The hook reads local JSONL session logs and extracts `usage.cost`, including an object-form `usage.cost.total`. It accumulates cost changes and transmits them as `cost_delta` to the external reporting API. The primary privacy documentation states that only Agent name, country, and token counts are shared. Model identifiers are described elsewhere, but monetary or usage-cost telemetry is not disclosed. Consequently, the hook's actual collection exceeds the documented privacy scope. After a gateway-startup event, the hook starts an interval that scans recent JSONL files every minute and reports accumulated deltas. Cost reporting is therefore automatic once the hook is activated. No message-content transmission was identified in the reviewed implementation. The ...[truncated 1050 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Remove `cost_delta` unless cost telemetry is essential to the declared leaderboard function. 2. If cost data is required, update the privacy statement and hook documentation before collection. 3. Obtain explicit, separate consent for transmitting monetary or cost-related metadata. 4. Add configuration controls such as: ```json { "report_tokens": true, "report_models": true, "report_cost": false } ``` 5. Default optional telemetry fields to disabled. 6. Align the hook and cron reporter payload schemas so that enabling one mechanism does not unexpectedly disclose more data than the other. 7. Log a concise local summary of the exact fields sent with each report. 8. Add tests that compare documented telemetry fields against generated outbound payloads. 9. Provide a documented way to delete or reset locally stored reporting state. ]]>

T09 · Insecure Skill Coding Practices

Note
Location
scripts/test.sh:18
Finding
Predictable Shared Temporary Files Allow Symlink-Based File Overwrite<![CDATA[ ## Vulnerability Details **File Location**: `scripts/test.sh:18-36` **Vulnerability Type**: Unsafe temporary-file handling **Risk Level**: Low ### Vulnerable Code ```bash echo -n "2. Report endpoint contract: " HTTP_CODE=$(curl -s -o /tmp/roc-test-report.json -w "%{http_code}" -X POST "$API_URL/report" \ -H "Content-Type: application/json" \ -d '{}') if [ "$HTTP_CODE" = "400" ] && grep -q "Missing required fields" /tmp/roc-test-report.json; then echo "OK" else echo "FAIL - unexpected response (HTTP $HTTP_CODE)" cat /tmp/roc-test-report.json fi echo -n "3. Rank endpoint reachable: " RANK_HTTP=$(curl -s -o /tmp/roc-test-rank.json -w "%{http_code}" "$API_URL/rank?agent=non-existent-agent") if [ "$RANK_HTTP" = "404" ] || [ "$RANK_HTTP" = "400" ]; then echo "OK" else echo "FAIL - unexpected response (HTTP $RANK_HTTP)" cat /tmp/roc-test-rank.json fi ``` ### Technical Analysis The test script writes HTTP responses to fixed paths in the globally shared `/tmp` directory. It does not create the files exclusively, verify ownership, reject symbolic links, or use a private temporary directory. On systems where the invoked `curl` follows symbolic links, another local user can pre-create one of these paths as a symbolic link to a file writable by the victim. Running `test.sh` can then truncate or overwrite that target with an HTTP response. The exploit is limited by the invoking user's filesystem permissions. The script does not use `sudo`, so it cannot overwrite root-owned files unless it is separately run with elevated privileges. ### Attack Path 1. A local attacker predicts the fixed filename `/tmp/roc-test-report.json` or `/tmp/roc-test-rank.json`. 2. The attacker creates that path as a symbolic link to a file writable by the victim. 3. The victim runs `scripts/test.sh`. 4. `curl -o` opens the predictable path and follows the symbolic link. 5. The target file is overwritten or truncated with the remote API response. 6. Depending on the s ...[truncated 472 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Create a private temporary directory with `mktemp -d`. 2. Register a cleanup trap immediately after creation. 3. Store all response files inside that directory. 4. Apply a restrictive umask before creating temporary data. 5. Do not run this test script with elevated privileges. Example hardening: ```bash umask 077 TMP_DIR="$(mktemp -d "${TMPDIR:-/tmp}/roc-test.XXXXXX")" trap 'rm -rf -- "$TMP_DIR"' EXIT REPORT_RESPONSE="$TMP_DIR/report.json" RANK_RESPONSE="$TMP_DIR/rank.json" HTTP_CODE=$(curl -s -o "$REPORT_RESPONSE" -w "%{http_code}" -X POST "$API_URL/report" \ -H "Content-Type: application/json" \ -d '{}') RANK_HTTP=$(curl -s -o "$RANK_RESPONSE" -w "%{http_code}" \ "$API_URL/rank?agent=non-existent-agent") ``` 6. Quote all temporary paths and fail if secure temporary-file creation is unavailable. ]]>
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Rogue AgentSelf-Modification, Session Persistence
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Findings (19)

Tp4

High
Category
MCP Tool Poisoning
Confidence
95% confidence
Finding
The skill description emphasizes a simple install and token reporting, but the documented behavior also includes persistent background execution, local file modification, and external telemetry submission. This mismatch is dangerous because users may consent to installation without realizing the full extent of ongoing data collection and persistence mechanisms.

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding
The skill description emphasizes a simple install and token reporting, but the documented behavior also includes persistent background execution, local file modification, and external telemetry submission. This mismatch is dangerous because users may consent to installation without realizing the full extent of ongoing data collection and persistence mechanisms.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding
The skill description emphasizes a simple install and token reporting, but the documented behavior also includes persistent background execution, local file modification, and external telemetry submission. This mismatch is dangerous because users may consent to installation without realizing the full extent of ongoing data collection and persistence mechanisms.

Description-Behavior Mismatch

High
Confidence
97% confidence
Finding
The handler periodically scans local session JSONL files, aggregates usage data, and transmits that telemetry to an external service at rankingofclaws.angelstreet.io. This is inconsistent with a benign 'local reporting' expectation and creates undisclosed exfiltration of operational metadata, including model usage, token counts, inferred activity timing, and host-linked identifiers.

Missing User Warnings

High
Confidence
96% confidence
Finding
The skill performs background telemetry uploads on startup and command events without any user-facing notice or runtime consent in this code path. Hidden background egress combined with persistent identifiers materially increases privacy risk and violates the principle of least surprise for an installable skill.

Description-Behavior Mismatch

High
Confidence
97% confidence
Finding
The script reads local OpenClaw session JSONL files, derives per-model token usage deltas, and sends that metadata to an external service at rankingofclaws.angelstreet.io. Although the skill description hints at reporting, the code performs outbound exfiltration of locally derived usage data and identifiers without meaningful consent, transparency, or controls, which is a real privacy and telemetry risk.

Lp3

Medium
Category
MCP Least Privilege
Confidence
87% confidence
Finding
The skill advertises and invokes shell execution via an installer script, and its documented behavior includes reading session files, writing config/state, and making network requests, but it does not declare any explicit tool scope or permissions. Missing scope declarations make it harder for users and policy enforcement to understand or constrain what the skill can access, increasing the chance of overprivileged execution.

External Transmission

Medium
Category
Data Exfiltration
Content
```bash
# Get leaderboard
curl https://rankingofclaws.angelstreet.io/api/leaderboard?limit=50

# Check your rank
curl https://rankingofclaws.angelstreet.io/api/rank?agent=MyAgent
Confidence
80% confidence
Finding
The skill explicitly transmits usage-related telemetry to an external service, including agent name, country, model, token deltas, and a gateway identifier hash. Even if no message content is sent, this creates privacy and operational metadata exposure, and in this skill's context it is more sensitive because it continuously derives data from local session files and reports it on a recurring basis.

Missing User Warnings

Medium
Confidence
96% confidence
Finding
The hook explicitly states it reads session JSONL logs and reports token counts plus model identifiers to an external service, but the description does not warn users that session-derived data is transmitted off-host. Even if the payload is framed as token deltas, model IDs and session telemetry can still reveal usage patterns, agent activity, and potentially sensitive operational metadata; omission of a clear warning prevents informed consent.

Context-Inappropriate Capability

Medium
Confidence
93% confidence
Finding
The code derives a persistent gateway ID from hostname and HOME and uses hostname as fallback agent identity, creating a stable host-linked identifier unrelated to simple token delta collection. Even though the raw hostname/HOME are hashed before transmission, the identifier still enables long-term tracking and correlation of a specific machine across reports.

Missing User Warnings

Medium
Confidence
90% confidence
Finding
The handler enumerates agent session directories and parses recent .jsonl session logs to extract assistant usage data without any disclosure in this file. Session files may contain sensitive operational metadata and, depending on format drift, could expose more than intended if the parser is later extended or reused.

Description-Behavior Mismatch

Medium
Confidence
93% confidence
Finding
The installer persists host-specific registration metadata to config.json, including a stable gateway identifier derived from hostname and HOME, and does so automatically during installation. This creates durable host identity and tracking behavior that goes beyond the stated purpose of merely reporting JSONL session deltas, which is risky because it enables persistent correlation of activity across runs and may surprise users or violate least-privilege expectations.

Context-Inappropriate Capability

Medium
Confidence
93% confidence
Finding
The fallback gateway_id is deterministically derived from the machine hostname and HOME path, creating a stable pseudonymous identifier that can track the same user or host across reports. This persistence is not necessary for simple token-delta reporting and increases privacy risk by enabling long-term correlation of activity.

Missing User Warnings

Medium
Confidence
95% confidence
Finding
The script performs silent HTTP POST requests containing gateway_id, agent name, country, model, and token deltas to a third-party API without any user-facing warning at runtime. In a skill that scans local session history, undisclosed outbound transmission makes the behavior more dangerous because users may reasonably expect local processing only.

Session Persistence

Medium
Category
Rogue Agent
Content
MARKER="# ranking-of-claws"
CRON_LINE="*/10 * * * * bash \"$REPORT_SCRIPT\" >> \"$LOG_FILE\" 2>&1 $MARKER"

CURRENT_CRON="$(crontab -l 2>/dev/null || true)"

if printf '%s\n' "$CURRENT_CRON" | grep -Fq "$MARKER"; then
  echo "ranking-of-claws: cron already configured."
Confidence
96% confidence
Finding
This script establishes persistence by installing a recurring cron job that executes every 10 minutes, causing code to run automatically outside the user's immediate action. Even though the apparent purpose is operational convenience, persistence is security-relevant because it creates ongoing execution, log growth, and a durable foothold that may continue after the user forgets it was installed.

Description-Behavior Mismatch

Medium
Confidence
95% confidence
Finding
The manifest describes a skill that registers once, sets up cron locally, and reports token/model deltas from JSONL sessions without editing a config file. This test script reaches out to a remote API service, probing multiple endpoints over the network, which is not reflected in that local-install/reporting description and expands the apparent behavior surface.

External Transmission

Medium
Category
Data Exfiltration
Content
# Test API connectivity
echo -n "1. API reachable: "
if curl -sf "$API_URL/stats" > /dev/null 2>&1; then
  echo "OK"
else
  echo "FAIL - cannot reach $API_URL"
Confidence
70% confidence
Finding
Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Missing User Warnings

Low
Confidence
89% confidence
Finding
The script silently writes persistent state under ~/.openclaw to track prior model totals and compute future deltas. While lower severity than network exfiltration, undisclosed persistence can surprise users, create forensic residue, and interact poorly with expectations for ephemeral tooling.

Intent-Code Divergence

Low
Confidence
96% confidence
Finding
The script prints 'All checks passed' unconditionally at the end, even though checks 2-4 only emit failure messages and do not exit nonzero. This can mislead users or automation into believing the setup is healthy when some validations actually failed, reducing trust in test results and potentially masking deployment or integration issues.

Static analysis

No suspicious patterns detected.