Back to skill

Security audit

Ranking Of Claws

Security checks for vulnerabilities and agentic risk

Overview

This leaderboard skill is mostly aligned with its stated purpose, but it installs ongoing reporting and transmits session-derived usage metadata with some important under-disclosed behavior.

Review carefully before installing. This skill will report OpenClaw usage metadata to an external leaderboard and can keep doing so through cron or hooks. Confirm the identity in config.json, remove or regenerate the bundled Pika configuration, and only enable cron/hooks if you are comfortable with recurring uploads of token, model, and possible cost metadata.

Vulnerability Patterns
  • System PersistenceInstalls backdoors, hooks, services, or scheduled tasks that survive the run
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (4)

T06 · System Persistence

Error
Location
scripts/setup-cron.sh:6
Finding

Automatic Installation of Persistent Scheduled Telemetry Reporting

Content
View full analysis
> \"$LOG_FILE\" 2>&1 $MARKER" CURRENT_CRON="$(crontab -l 2>/dev/null || true)" if printf '%s\n' "$CURRENT_CRON" | grep -Fq "$MARKER"; then echo "ranking-of-claws: cron already configured." exit 0 fi { printf '%s\n' "$CURRENT_CRON" printf '%s\n' "$CRON_LINE" } | sed '/^[[:space:]]*$/N;/^\n$/D' | crontab - ``` The installer invokes this code automatically: ```bash if [ "$FORCE" != "1" ] && [ -n "${EXISTING_NAME:-}" ]; then echo "ranking-of-claws: already registered as \"$EXISTING_NAME\" (config.json kept)." echo "To re-register: ROC_FORCE_REREGISTER=1 bash scripts/install.sh" bash "$SCRIPT_DIR/setup-cron.sh" exit 0 fi ``` ```bash bash "$SCRIPT_DIR/setup-cron.sh" ``` ### Technical Analysis The installation process modifies the current user's crontab and registers `report.sh` to run every ten minutes. The scheduled task survives the installer process, shell termination, Agent sessions, and system reboots where cron is enabled. The recurring script reads OpenClaw session files from `~/.openclaw/agents/*/sessions/*.jsonl`, derives token and model usage, and sends the resulting deltas to an external API. Although automatic cron installation is disclosed in `SKILL.md`, persistent execution is not the minimum privilege required for manual leaderboard reporting. No separate consent prompt or opt-in control is presented before modifying the crontab. The script preserves existing crontab content and uses a fixed marker to avoid duplicate registration. However, the project does not provide an uninstall procedure to remove the marked entry. Deleting the Skill ...[truncated 1154 chars]
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Warning
Location
config.json:1
Finding

Bundled Registration Identity Silently Bypasses User Registration

Content
View full analysis
Remediation
View remediation

other

Warning
Location
hooks/handler.js:128
Finding

Hook Sends Undisclosed Cost Telemetry to an External Service

Content
View full analysis
0 ? delta.tokens : delta.input + delta.output, tokens_in_delta: delta.input, tokens_out_delta: delta.output, cost_delta: delta.cost, model, }); ``` The privacy statement is narrower than the actual hook payload: ```markdown ## Privacy - Only agent name, country, and token counts are shared - No message content transmitted - Gateway ID is a non-reversible hash ``` ### Technical Analysis The hook reads local JSONL session logs and extracts `usage.cost`, including an object-form `usage.cost.total`. It accumulates cost changes and transmits them as `cost_delta` to the external reporting API. The primary privacy documentation states that only Agent name, country, and token counts are shared. Model identifiers are described elsewhere, but monetary or usage-cost telemetry is not disclosed. Consequently, the hook's actual collection exceeds the documented privacy scope. After a gateway-startup event, the hook starts an interval that scans recent JSONL files every minute and reports accumulated deltas. Cost reporting is therefore automatic once the hook is activated. No message-content transmission was identified in the reviewed implementation. The ...[truncated 1050 chars]
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Note
Location
scripts/test.sh:18
Finding

Predictable Shared Temporary Files Allow Symlink-Based File Overwrite

Content
View full analysis
Remediation
View remediation
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Rogue AgentSelf-Modification, Session Persistence
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Findings (19)

Tp4

High
Category
MCP Tool Poisoning
Confidence
95% confidence
Finding

The skill description emphasizes a simple install and token reporting, but the documented behavior also includes persistent background execution, local file modification, and external telemetry submission. This mismatch is dangerous because users may consent to installation without realizing the full extent of ongoing data collection and persistence mechanisms.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding

The skill description emphasizes a simple install and token reporting, but the documented behavior also includes persistent background execution, local file modification, and external telemetry submission. This mismatch is dangerous because users may consent to installation without realizing the full extent of ongoing data collection and persistence mechanisms.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The skill description emphasizes a simple install and token reporting, but the documented behavior also includes persistent background execution, local file modification, and external telemetry submission. This mismatch is dangerous because users may consent to installation without realizing the full extent of ongoing data collection and persistence mechanisms.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The handler periodically scans local session JSONL files, aggregates usage data, and transmits that telemetry to an external service at rankingofclaws.angelstreet.io. This is inconsistent with a benign 'local reporting' expectation and creates undisclosed exfiltration of operational metadata, including model usage, token counts, inferred activity timing, and host-linked identifiers.

Content

No source excerpt is available for this finding.

Missing User Warnings

High
Category
Not specified by scanner
Confidence
96% confidence
Finding

The skill performs background telemetry uploads on startup and command events without any user-facing notice or runtime consent in this code path. Hidden background egress combined with persistent identifiers materially increases privacy risk and violates the principle of least surprise for an installable skill.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The script reads local OpenClaw session JSONL files, derives per-model token usage deltas, and sends that metadata to an external service at rankingofclaws.angelstreet.io. Although the skill description hints at reporting, the code performs outbound exfiltration of locally derived usage data and identifiers without meaningful consent, transparency, or controls, which is a real privacy and telemetry risk.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
87% confidence
Finding

The skill advertises and invokes shell execution via an installer script, and its documented behavior includes reading session files, writing config/state, and making network requests, but it does not declare any explicit tool scope or permissions. Missing scope declarations make it harder for users and policy enforcement to understand or constrain what the skill can access, increasing the chance of overprivileged execution.

Content

No source excerpt is available for this finding.

External Transmission

Medium
Category
Data Exfiltration
Confidence
80% confidence
Finding

The skill explicitly transmits usage-related telemetry to an external service, including agent name, country, model, token deltas, and a gateway identifier hash. Even if no message content is sent, this creates privacy and operational metadata exposure, and in this skill's context it is more sensitive because it continuously derives data from local session files and reports it on a recurring basis.

Content

Scanner excerpt · SKILL.md (reported line 72)May include surrounding context.

bash
# Get leaderboard
curl https://rankingofclaws.angelstreet.io/api/leaderboard?limit=50

# Check your rank
curl https://rankingofclaws.angelstreet.io/api/rank?agent=MyAgent

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The hook explicitly states it reads session JSONL logs and reports token counts plus model identifiers to an external service, but the description does not warn users that session-derived data is transmitted off-host. Even if the payload is framed as token deltas, model IDs and session telemetry can still reveal usage patterns, agent activity, and potentially sensitive operational metadata; omission of a clear warning prevents informed consent.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The code derives a persistent gateway ID from hostname and HOME and uses hostname as fallback agent identity, creating a stable host-linked identifier unrelated to simple token delta collection. Even though the raw hostname/HOME are hashed before transmission, the identifier still enables long-term tracking and correlation of a specific machine across reports.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The handler enumerates agent session directories and parses recent .jsonl session logs to extract assistant usage data without any disclosure in this file. Session files may contain sensitive operational metadata and, depending on format drift, could expose more than intended if the parser is later extended or reused.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The installer persists host-specific registration metadata to config.json, including a stable gateway identifier derived from hostname and HOME, and does so automatically during installation. This creates durable host identity and tracking behavior that goes beyond the stated purpose of merely reporting JSONL session deltas, which is risky because it enables persistent correlation of activity across runs and may surprise users or violate least-privilege expectations.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The fallback gateway_id is deterministically derived from the machine hostname and HOME path, creating a stable pseudonymous identifier that can track the same user or host across reports. This persistence is not necessary for simple token-delta reporting and increases privacy risk by enabling long-term correlation of activity.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The script performs silent HTTP POST requests containing gateway_id, agent name, country, model, and token deltas to a third-party API without any user-facing warning at runtime. In a skill that scans local session history, undisclosed outbound transmission makes the behavior more dangerous because users may reasonably expect local processing only.

Content

No source excerpt is available for this finding.

Session Persistence

Medium
Category
Rogue Agent
Confidence
96% confidence
Finding

This script establishes persistence by installing a recurring cron job that executes every 10 minutes, causing code to run automatically outside the user's immediate action. Even though the apparent purpose is operational convenience, persistence is security-relevant because it creates ongoing execution, log growth, and a durable foothold that may continue after the user forgets it was installed.

Content

Scanner excerpt · scripts/setup-cron.sh (reported line 10)May include surrounding context.

sh
MARKER="# ranking-of-claws"
CRON_LINE="*/10 * * * * bash \"$REPORT_SCRIPT\" >> \"$LOG_FILE\" 2>&1 $MARKER"

CURRENT_CRON="$(crontab -l 2>/dev/null || true)"

if printf '%s\n' "$CURRENT_CRON" | grep -Fq "$MARKER"; then
  echo "ranking-of-claws: cron already configured."

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The manifest describes a skill that registers once, sets up cron locally, and reports token/model deltas from JSONL sessions without editing a config file. This test script reaches out to a remote API service, probing multiple endpoints over the network, which is not reflected in that local-install/reporting description and expands the apparent behavior surface.

Content

No source excerpt is available for this finding.

External Transmission

Medium
Category
Data Exfiltration
Confidence
70% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · scripts/test.sh (reported line 10)May include surrounding context.

sh
# Test API connectivity
echo -n "1. API reachable: "
if curl -sf "$API_URL/stats" > /dev/null 2>&1; then
  echo "OK"
else
  echo "FAIL - cannot reach $API_URL"

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
89% confidence
Finding

The script silently writes persistent state under ~/.openclaw to track prior model totals and compute future deltas. While lower severity than network exfiltration, undisclosed persistence can surprise users, create forensic residue, and interact poorly with expectations for ephemeral tooling.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Low
Category
Not specified by scanner
Confidence
96% confidence
Finding

The script prints 'All checks passed' unconditionally at the end, even though checks 2-4 only emit failure messages and do not exit nonzero. This can mislead users or automation into believing the setup is healthy when some validations actually failed, reducing trust in test results and potentially masking deployment or integration issues.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.