API Benchmark

• Outcome: Review
• Latest audit: Sep 10, 2026 · 9:04 PM UTC
• Version: 1.0.2

Overview

This benchmarking skill is mostly coherent, but it can send API keys and prompts to unvalidated provider URLs, including plaintext HTTP endpoints, so users should review configuration carefully before use.

Install only if you are comfortable with a local Python script reading your OpenCLAW provider configuration and sending test prompts plus API credentials to the configured endpoints. Use environment-variable API keys, test or limited-scope keys, spending limits, and HTTPS provider URLs only; avoid sensitive custom prompts and treat --dry-run as unsafe until the preflight behavior is fixed.

A.I.G

Vulnerability Patterns

• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review
• Embedded Malicious Code: Ships malicious scripts inside the skill and executes them locally

Findings (1)

T09 · Insecure Skill Coding Practices

Warning

• Location: main.py:150

Finding

API Credentials May Be Transmitted over Unencrypted HTTP

Content

View full analysis

Vulnerability Details

File Location: main.py:150, main.py:216-218, main.py:231-233, main.py:284-287, and main.py:337-340
Vulnerability Type: Plaintext transmission of sensitive credentials
Risk Level: Medium

The provider URL is loaded directly from configuration without requiring HTTPS:

python

base_url = provider.get("baseUrl", "")

The unvalidated URL is subsequently used for authenticated Anthropic and OpenAI-compatible requests:

python

headers = {"x-api-key": target.api_key, "anthropic-version": "2023-06-01", "content-type": "application/json"}
payload = {"model": target.model, "max_tokens": 256, "stream": False, "messages": [{"role": "user", "content": prompt}]}
r = requests.post(f"{target.base_url}/v1/messages", headers=headers, json=payload, timeout=30)

python

headers = {"Authorization": f"Bearer {target.api_key}", "content-type": "application/json"}
payload = {"model": target.model, "max_tokens": 32, "stream": False, "messages": [{"role": "user", "content": prompt}]}
r = requests.post(f"{target.base_url}/chat/completions", headers=headers, json=payload, timeout=20)

python

headers = {"x-api-key": target.api_key, "anthropic-version": "2023-06-01", "content-type": "application/json"}
payload = {"model": target.model, "max_tokens": max_tokens, "stream": True, "messages": [{"role": "user", "content": prompt}]}

t_start = time.perf_counter()

The corresponding Anthropic streaming request is:

python

with requests.post(f"{target.base_url}/v1/messages", headers=headers, json=payload, stream=True, timeout=timeout) as resp:

The OpenAI-compatible streaming request has the same issue:

python

headers = {"Authorization": f"Bearer {target.api_key}", "content-type": "application/json"}
payload = {"model": target.model, "max_tokens": max_tokens, "stream": True, "messages": [{"role": "user", "content": prompt}]}

t_start = time.perf_counter(
...[truncated 2640 chars]

Remediation

View remediation

Remediation Suggestions

1. Parse every provider URL with urllib.parse.urlparse and reject malformed URLs.
2. Require the https scheme for all remote provider endpoints before constructing a Target.
3. If plaintext transport is needed for local development, permit it only through an explicit opt-in flag and only for loopback destinations such as 127.0.0.1, ::1, or localhost. Display a prominent warning when this exception is used.
4. Reject URLs containing embedded user information, unexpected fragments, or ambiguous host representations.
5. Disable automatic redirects for authenticated requests with allow_redirects=False, or validate every redirect target before following it. Require the destination to remain on HTTPS and within the intended trusted origin.
6. Fail closed before reading or attaching the API key if URL validation fails.
7. Document that configuration files containing literal API keys must have restrictive filesystem permissions, while continuing to recommend environment-variable placeholders.
8. Add tests confirming rejection of http:// remote endpoints, malformed URLs, HTTPS-to-HTTP redirects, and attacker-controlled redirect destinations.
9. Use provider keys with minimum required permissions, spending limits, and rotation procedures to reduce the impact of accidental disclosure.

SkillSpector

Vulnerability Patterns

• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• MCP Least Privilege: Underdeclared Capability, Wildcard Permission, Missing Permission Declaration
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access
• Supply Chain: Unpinned Dependencies, External Script Fetching, Obfuscated Code

Findings (6)

Undeclared Tool Scope

Medium

• Category: MCP Least Privilege
• Confidence: 93% confidence

Finding

The skill performs network access and reads environment/configuration data, but it does not declare an explicit tool scope such as permissions or allowed-tools. That weakens user and platform visibility into what the skill can access, increasing the chance that sensitive configuration or API credentials are used without clear consent boundaries.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 95% confidence

Finding

The usage description does not clearly warn users that benchmark and connectivity-check operations will send prompts, model identifiers, and authentication-bearing requests to external API providers. In a benchmarking skill, this context makes the issue more significant because contacting third-party endpoints is the core behavior, so users may unintentionally expose sensitive prompts or use production credentials without realizing the data leaves the local environment.

Content

No source excerpt is available for this finding.

External Transmission

Medium

• Category: Data Exfiltration
• Confidence: 80% confidence

Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · main.py (reported line 161)May include surrounding context.

python

def _preflight_anthropic(target: Target, prompt: str) -> tuple:
    headers = {"x-api-key": target.api_key, "anthropic-version": "2023-06-01", "content-type": "application/json"}
    payload = {"model": target.model, "max_tokens": 256, "stream": False, "messages": [{"role": "user", "content": prompt}]}
    r = requests.post(f"{target.base_url}/v1/messages", headers=headers, json=payload, timeout=30)
    if r.status_code != 200:
        return False, f"HTTP {r.status_code}: {r.text[:200]}"
    data = r.json()

External Transmission

Medium

• Category: Data Exfiltration
• Confidence: 80% confidence

Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · main.py (reported line 239)May include surrounding context.

python

def _preflight_anthropic(target: Target, prompt: str) -> tuple:
    headers = {"x-api-key": target.api_key, "anthropic-version": "2023-06-01", "content-type": "application/json"}
    payload = {"model": target.model, "max_tokens": 256, "stream": False, "messages": [{"role": "user", "content": prompt}]}
    r = requests.post(f"{target.base_url}/v1/messages", headers=headers, json=payload, timeout=30)
    if r.status_code != 200:
        return False, f"HTTP {r.status_code}: {r.text[:200]}"
    data = r.json()

External Transmission

Medium

• Category: Data Exfiltration
• Confidence: 80% confidence

Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · main.py (reported line 178)May include surrounding context.

python

def _preflight_openai(target: Target, prompt: str) -> tuple:
    headers = {"Authorization": f"Bearer {target.api_key}", "content-type": "application/json"}
    payload = {"model": target.model, "max_tokens": 32, "stream": False, "messages": [{"role": "user", "content": prompt}]}
    r = requests.post(f"{target.base_url}/chat/completions", headers=headers, json=payload, timeout=20)
    if r.status_code != 200:
        return False, f"HTTP {r.status_code}: {r.text[:200]}"
    data = r.json()

External Transmission

Medium

• Category: Data Exfiltration
• Confidence: 80% confidence

Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · main.py (reported line 299)May include surrounding context.

python

def _preflight_openai(target: Target, prompt: str) -> tuple:
    headers = {"Authorization": f"Bearer {target.api_key}", "content-type": "application/json"}
    payload = {"model": target.model, "max_tokens": 32, "stream": False, "messages": [{"role": "user", "content": prompt}]}
    r = requests.post(f"{target.base_url}/chat/completions", headers=headers, json=payload, timeout=20)
    if r.status_code != 200:
        return False, f"HTTP {r.status_code}: {r.text[:200]}"
    data = r.json()