Back to skill

Security audit

OpenClaw Benchmark

Security checks for vulnerabilities and agentic risk

Overview

This benchmark skill is mostly transparent and purpose-aligned, but its report generator can render untrusted metric text as HTML.

Install only if you are comfortable with a benchmark that runs fixed shell and network tests and saves reports locally. Do not generate reports from metrics JSON you do not trust until the HTML escaping issue is fixed, and avoid sharing baseline files if model or system metadata is sensitive.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Warning
Location
score.py:56
Finding

Unescaped Metrics Data Allows HTML and Script Injection in Generated Reports

Content
View full analysis
{t["failed"]}失败' tests_html += f""" {t.get('id','')} {t.get('name','')} {dur}s {tokens//1000}k {tools} {status_text}{extra} """ ``` Other unescaped values are interpolated into the report in the system and model sections: ```python
{datetime.now().strftime('%Y-%m-%d %H:%M')} · {sys_info.get('os','')} · {sys_info.get('arch','')}
``` ```python
Model{model_info.get('name','—')}
Context Window{model_info.get('context_window','—')}
``` ```python
OpenClaw{sys_info.get('openclaw_version','—')}
Node.js{sys_info.get('node_version','—')}
``` ### Technical Analysis The scorer reads metrics from a JSON file supplied through the command line: ```python with open(sys.argv[1]) as f: metrics = json.load(f) ``` Multiple string values from that JSON object are then inserted directly into an HTML document using Python f-strings. No HTML escaping, sanitization, schema validation ...[truncated 2307 chars]
Remediation
View remediation
{test_id} {test_name} {dur}s {tokens // 1000}k {tools} {status_text}{failed} """ ``` 2. Escape all system and model metadata, including OS, architecture, version, provider, model name, and context-window values. 3. Prefer an HTML template engine with automatic escaping enabled rather than constructing the document through large f-strings. 4. Validate the metrics JSON against a strict schema: - Require numeric types for durations, token counts, ratios, and tool counts. - Restrict status values to an explicit allowlist such as `ok` and `failed`. - Set reasonable maximum lengths for all display strings. - Reject unexpected nested objects and arrays where scalar values are expected. 5. Add a restrictive Content Security Policy to reduce the effect of any missed escaping: ```html ``` If external resources are not required, do not permit scripts, remote images, frames, forms, or network connections. 6. Add regression tests using payloads containing characters and elements such as `<`, `>`, `"`, `'`, `
Vulnerability Patterns
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Tool MisuseTool Parameter Abuse, Chaining Abuse, Unsafe Defaults
  • Rogue AgentSelf-Modification, Session Persistence
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (9)

Tool Parameter Abuse

High
Category
Tool Misuse
Confidence
90% confidence
Finding

The skill includes a direct exec instruction to run 'rm /tmp/openclaw_bench.txt'. While the specific command targets a temporary file, destructive shell primitives like rm materially increase risk because they rely on exact parameter integrity and normalize deletion via unrestricted exec. If the command string is altered, templated, or generalized by an agent, it could delete unintended files or be repurposed for broader destructive actions.

Content

Scanner excerpt · SKILL.md (reported line 108)May include surrounding context.

依次执行以下操作,每步完成后记录结果:

  1. 用 exec 执行: echo "benchmark test $(date +%s)" > /tmp/openclaw_bench.txt
  2. 用 read 读取 /tmp/openclaw_bench.txt 的内容
  3. 用 exec 执行: rm /tmp/openclaw_bench.txt 把每步的操作和结果写入报告。
text

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

Several benchmark prompts require the subagent to answer in Chinese, and much of the operational text and schema labels are also Chinese-only. This imposes a specific language/locale behavior without user opt-in or a documented justification that the skill is intended only for Chinese-language benchmarking.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The skill requires external network access through web_search and web_fetch as part of the benchmark, but this capability is not tightly justified or bounded beyond measuring latency and fallback behavior. Network-enabled skills can leak contextual information, trigger requests to third-party services, and produce variable behavior influenced by remote content. In a benchmarking context, this is especially risky because users may treat the skill as routine diagnostics and overlook the privacy and egress implications.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The benchmark instructs the agent to use shell execution for steps that are not strictly necessary to compute a performance score, including creating and deleting files and querying local tool versions. Granting exec capability expands the attack surface and can normalize command execution in a skill that otherwise appears to be a measurement workflow. If the skill is run automatically or by a less cautious agent, these commands could be modified or extended to perform unintended local actions.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The protocol instructs the agent to create, read, and delete a file in /tmp without any clear warning that the skill will modify the local filesystem. Even though /tmp is temporary, these are still state-changing operations and establish a precedent for silent local writes. In environments where agents execute tools directly, lack of disclosure can surprise users and may violate least-surprise or policy expectations.

Content

No source excerpt is available for this finding.

Session Persistence

Medium
Category
Rogue Agent
Confidence
84% confidence
Finding

The benchmark writes session-derived metrics to /tmp/bench_metrics.json and then encourages saving baseline files for later comparison, creating session persistence beyond the immediate run. Although this is part of the benchmark design, persisted artifacts can retain environment metadata and performance history that may be sensitive in shared or managed systems. The danger is contextual: a diagnostic skill has a legitimate reason to persist data, but it should not do so silently or indefinitely.

Content

Scanner excerpt · SKILL.md (reported line 141)May include surrounding context.

md
---

## Step 7: Write Metrics & Compute Score

Write all metrics to `/tmp/bench_metrics.json`:

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The skill directs persistent writes of benchmark metrics, HTML reports, and baseline JSON files to /tmp and the user's Downloads directory without a prominent user-facing warning. Persistent local writes can expose system details, model information, and historical benchmarking data, and may create compliance or privacy issues if users do not realize artifacts are being saved. The risk is elevated because the files are intended to accumulate over time.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
95% confidence
Finding

The generated HTML explicitly sets lang="zh-CN" and contains substantial Chinese-only UI text, while the script does not provide any option for the user to select language or locale. This is a natural-language policy concern because it forces a specific locale experience without documented opt-in or justification.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
89% confidence
Finding

This code creates a directory under the user's Downloads folder and writes an HTML report there, but there is no confirmation prompt or prior disclosure before the write occurs. For a code file, file creation is a safety-relevant operation that should have some visible user disclosure when it is not clearly announced before execution.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.