ClawHub
Back to skill

Security audit

Model Throughput Tester

Security checks for vulnerabilities and agentic risk

Overview

This benchmarking skill is purpose-aligned, but API mode needs review because it can send your API key and prompts to any configured endpoint, including insecure HTTP.

Prefer auto mode when possible. For API mode, use only trusted HTTPS endpoints, avoid putting real keys in shell history, avoid sensitive benchmark prompts, and remember that generated reports can store the prompt and API URL locally.

A.I.G

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (2)

T09 · Insecure Skill Coding Practices

Warning
Location
throughput.py:353
Finding

API Key Accepted Through a Command-Line Argument

Content
View full analysis
/cmdline`, or system monitoring agents. - Terminal session logging and command auditing. - CI/CD job output or wrapper scripts that print invoked commands. - Diagnostic reports that capture process arguments. The key is legitimately required for direct API benchmarking, and the code does not intentionally transmit it anywhere except the selected endpoint. Nevertheless, accepting it directly through `argv` exposes the credential more broadly than necessary. ### Attack Path 1. A user follows the documented API-mode example and invokes the script with `--key `. 2. The shell records the command in its history, or the operating system exposes the argument in process metadata while the benchmark is running. 3. Another local account, monitoring process, log collector, or later shell-history reader obtains the key. 4. The attacker uses the recovered key directly against the corresponding API provider. 5. The attack ...[truncated 785 chars]
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Error
Location
throughput.py:188
Finding

Bearer API Credentials May Be Transmitted Over Plaintext HTTP

Content
View full analysis
Remediation
View remediation

SkillSpector

Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Findings (20)

External Transmission

Medium
Category
Data Exfiltration
Confidence
60% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · README.md (reported line 23)May include surrounding context.

md
def test_url_v1_suffix(t):
    """以 /v1 结尾 → 自动补 /chat/completions"""
    url = _build_api_url("https://api.openai.com/v1")
    if url == "https://api.openai.com/v1/chat/completions":
        t.ok("URL: /v1 后缀")
    else:

External Transmission

Medium
Category
Data Exfiltration
Confidence
60% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · README.zh.md (reported line 101)May include surrounding context.

md
def test_url_v1_suffix(t):
    """以 /v1 结尾 → 自动补 /chat/completions"""
    url = _build_api_url("https://api.openai.com/v1")
    if url == "https://api.openai.com/v1/chat/completions":
        t.ok("URL: /v1 后缀")
    else:

External Transmission

Medium
Category
Data Exfiltration
Confidence
60% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 165)May include surrounding context.

md
def test_url_v1_suffix(t):
    """以 /v1 结尾 → 自动补 /chat/completions"""
    url = _build_api_url("https://api.openai.com/v1")
    if url == "https://api.openai.com/v1/chat/completions":
        t.ok("URL: /v1 后缀")
    else:

External Transmission

Medium
Category
Data Exfiltration
Confidence
60% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · tests/test_throughput.py (reported line 177)May include surrounding context.

python
def test_url_v1_suffix(t):
    """以 /v1 结尾 → 自动补 /chat/completions"""
    url = _build_api_url("https://api.openai.com/v1")
    if url == "https://api.openai.com/v1/chat/completions":
        t.ok("URL: /v1 后缀")
    else:

External Transmission

Medium
Category
Data Exfiltration
Confidence
60% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · tests/test_throughput.py (reported line 178)May include surrounding context.

python
def test_url_v1_suffix(t):
    """以 /v1 结尾 → 自动补 /chat/completions"""
    url = _build_api_url("https://api.openai.com/v1")
    if url == "https://api.openai.com/v1/chat/completions":
        t.ok("URL: /v1 后缀")
    else:

External Transmission

Medium
Category
Data Exfiltration
Confidence
60% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · README.zh.md (reported line 37)May include surrounding context.

md
def test_url_no_suffix(t):
    url = _build_api_url("https://api.example.com")
    if url == "https://api.example.com/v1/chat/completions":
        t.ok("URL: 无后缀 → 加 /v1")
    else:
        t.fail("URL 无后缀", f"得到 {url}")

External Transmission

Medium
Category
Data Exfiltration
Confidence
60% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 72)May include surrounding context.

md
def test_url_no_suffix(t):
    url = _build_api_url("https://api.example.com")
    if url == "https://api.example.com/v1/chat/completions":
        t.ok("URL: 无后缀 → 加 /v1")
    else:
        t.fail("URL 无后缀", f"得到 {url}")

External Transmission

Medium
Category
Data Exfiltration
Confidence
60% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · tests/test_throughput.py (reported line 194)May include surrounding context.

python
def test_url_no_suffix(t):
    url = _build_api_url("https://api.example.com")
    if url == "https://api.example.com/v1/chat/completions":
        t.ok("URL: 无后缀 → 加 /v1")
    else:
        t.fail("URL 无后缀", f"得到 {url}")

External Transmission

Medium
Category
Data Exfiltration
Confidence
60% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · tests/test_throughput.py (reported line 202)May include surrounding context.

python
def test_url_no_suffix(t):
    url = _build_api_url("https://api.example.com")
    if url == "https://api.example.com/v1/chat/completions":
        t.ok("URL: 无后缀 → 加 /v1")
    else:
        t.fail("URL 无后缀", f"得到 {url}")

External Transmission

Medium
Category
Data Exfiltration
Confidence
60% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · tests/test_throughput.py (reported line 203)May include surrounding context.

python
def test_url_no_suffix(t):
    url = _build_api_url("https://api.example.com")
    if url == "https://api.example.com/v1/chat/completions":
        t.ok("URL: 无后缀 → 加 /v1")
    else:
        t.fail("URL 无后缀", f"得到 {url}")

External Transmission

Medium
Category
Data Exfiltration
Confidence
60% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · tests/test_throughput.py (reported line 211)May include surrounding context.

python
def test_url_no_suffix(t):
    url = _build_api_url("https://api.example.com")
    if url == "https://api.example.com/v1/chat/completions":
        t.ok("URL: 无后缀 → 加 /v1")
    else:
        t.fail("URL 无后缀", f"得到 {url}")

External Transmission

Medium
Category
Data Exfiltration
Confidence
60% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · tests/test_throughput.py (reported line 212)May include surrounding context.

python
def test_url_no_suffix(t):
    url = _build_api_url("https://api.example.com")
    if url == "https://api.example.com/v1/chat/completions":
        t.ok("URL: 无后缀 → 加 /v1")
    else:
        t.fail("URL 无后缀", f"得到 {url}")

External Transmission

Medium
Category
Data Exfiltration
Confidence
60% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · tests/test_throughput.py (reported line 258)May include surrounding context.

python
{"model": "model-b", "iter": 1, "elapsed": 2.0, "output_tokens": 100, "tokens_per_sec": 50.0, "status": "ok", "error": ""},
        ],
    }
    report = throughput.build_markdown_report(results, "api", "https://api.test/v1", 1, "test")
    if "model-a" in report and "model-b" in report and "100.0" in report and "50.0" in report:
        t.ok("report: 多模型 dict 输入")
    else:

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · throughput.py (reported line 96)May include surrounding context.

python
"--thinking", "off"
        ]
        try:
            subprocess.run(
                warmup_cmd,
                stdout=subprocess.PIPE,
                stderr=subprocess.STDOUT,

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · throughput.py (reported line 121)May include surrounding context.

python
start = time.perf_counter()
            # Use bytes mode with stderr merged to stdout to ensure JSON output
            # (capture_output=True + text=True causes openclaw to emit human-readable format)
            r = subprocess.run(
                cmd,
                stdout=subprocess.PIPE,
                stderr=subprocess.STDOUT,

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The API mode sends user-provided prompts and a bearer token to an arbitrary URL without any trust validation, destination allowlisting, or explicit warning that sensitive content and credentials will be transmitted. In a skill context, this can expose secrets, internal prompts, or API keys to an attacker-controlled endpoint if the URL is misconfigured or influenced by untrusted input.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
84% confidence
Finding

This is a markdown file, so SQP-2 applies to omissions in the skill description. The README explains API mode and generated report files, but it does not explicitly warn that custom prompts are sent to the configured remote API endpoint and that benchmark results are written to local files, which could affect privacy or user data handling.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Low
Category
Not specified by scanner
Confidence
85% confidence
Finding

The top-level docstring says this is a '纯逻辑单测（不发起真实请求）', implying the test is isolated from real side effects. However, the test imports the production module and directly calls throughput.get_current_model() later, which can perform real configuration/file reads depending on that implementation, so the documentation overstates the isolation of the test.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
89% confidence
Finding

This Python test file includes user-facing usage text and extensive console output in Chinese, such as the module docstring and printed test section headings/results. The policy requires flagging language or locale constraints when a skill forces a specific language without user opt-in, and this file does not offer any language selection or document a justified locale restriction.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
76% confidence
Finding

Auto mode accesses ~/.openclaw/openclaw.json to detect the current model and provider settings. Although this supports the tool's purpose, the file access is not disclosed in a user-facing warning, comment, or prompt, so users may not realize local configuration is being inspected.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.