Back to skill

Security audit

API Benchmark

Security checks for vulnerabilities and agentic risk

Overview

This benchmarking skill is mostly coherent, but it can send API keys and prompts to unvalidated provider URLs, including plaintext HTTP endpoints, so users should review configuration carefully before use.

Install only if you are comfortable with a local Python script reading your OpenCLAW provider configuration and sending test prompts plus API credentials to the configured endpoints. Use environment-variable API keys, test or limited-scope keys, spending limits, and HTTPS provider URLs only; avoid sensitive custom prompts and treat --dry-run as unsafe until the preflight behavior is fixed.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Warning
Location
main.py:150
Finding
API Credentials May Be Transmitted over Unencrypted HTTP## Vulnerability Details **File Location**: `main.py:150`, `main.py:216-218`, `main.py:231-233`, `main.py:284-287`, and `main.py:337-340` **Vulnerability Type**: Plaintext transmission of sensitive credentials **Risk Level**: Medium The provider URL is loaded directly from configuration without requiring HTTPS: ```python base_url = provider.get("baseUrl", "") ``` The unvalidated URL is subsequently used for authenticated Anthropic and OpenAI-compatible requests: ```python headers = {"x-api-key": target.api_key, "anthropic-version": "2023-06-01", "content-type": "application/json"} payload = {"model": target.model, "max_tokens": 256, "stream": False, "messages": [{"role": "user", "content": prompt}]} r = requests.post(f"{target.base_url}/v1/messages", headers=headers, json=payload, timeout=30) ``` ```python headers = {"Authorization": f"Bearer {target.api_key}", "content-type": "application/json"} payload = {"model": target.model, "max_tokens": 32, "stream": False, "messages": [{"role": "user", "content": prompt}]} r = requests.post(f"{target.base_url}/chat/completions", headers=headers, json=payload, timeout=20) ``` ```python headers = {"x-api-key": target.api_key, "anthropic-version": "2023-06-01", "content-type": "application/json"} payload = {"model": target.model, "max_tokens": max_tokens, "stream": True, "messages": [{"role": "user", "content": prompt}]} t_start = time.perf_counter() ``` The corresponding Anthropic streaming request is: ```python with requests.post(f"{target.base_url}/v1/messages", headers=headers, json=payload, stream=True, timeout=timeout) as resp: ``` The OpenAI-compatible streaming request has the same issue: ```python headers = {"Authorization": f"Bearer {target.api_key}", "content-type": "application/json"} payload = {"model": target.model, "max_tokens": max_tokens, "stream": True, "messages": [{"role": "user", "content": prompt}]} t_start = time.perf_counter( ...[truncated 2640 chars]
Remediation
## Remediation Suggestions 1. Parse every provider URL with `urllib.parse.urlparse` and reject malformed URLs. 2. Require the `https` scheme for all remote provider endpoints before constructing a `Target`. 3. If plaintext transport is needed for local development, permit it only through an explicit opt-in flag and only for loopback destinations such as `127.0.0.1`, `::1`, or `localhost`. Display a prominent warning when this exception is used. 4. Reject URLs containing embedded user information, unexpected fragments, or ambiguous host representations. 5. Disable automatic redirects for authenticated requests with `allow_redirects=False`, or validate every redirect target before following it. Require the destination to remain on HTTPS and within the intended trusted origin. 6. Fail closed before reading or attaching the API key if URL validation fails. 7. Document that configuration files containing literal API keys must have restrictive filesystem permissions, while continuing to recommend environment-variable placeholders. 8. Add tests confirming rejection of `http://` remote endpoints, malformed URLs, HTTPS-to-HTTP redirects, and attacker-controlled redirect destinations. 9. Use provider keys with minimum required permissions, spending limits, and rotation procedures to reduce the impact of accidental disclosure.
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
Findings (6)

Lp3

Medium
Category
MCP Least Privilege
Confidence
93% confidence
Finding
The skill performs network access and reads environment/configuration data, but it does not declare an explicit tool scope such as permissions or allowed-tools. That weakens user and platform visibility into what the skill can access, increasing the chance that sensitive configuration or API credentials are used without clear consent boundaries.

Missing User Warnings

Medium
Confidence
95% confidence
Finding
The usage description does not clearly warn users that benchmark and connectivity-check operations will send prompts, model identifiers, and authentication-bearing requests to external API providers. In a benchmarking skill, this context makes the issue more significant because contacting third-party endpoints is the core behavior, so users may unintentionally expose sensitive prompts or use production credentials without realizing the data leaves the local environment.

External Transmission

Medium
Category
Data Exfiltration
Content
def _preflight_anthropic(target: Target, prompt: str) -> tuple:
    headers = {"x-api-key": target.api_key, "anthropic-version": "2023-06-01", "content-type": "application/json"}
    payload = {"model": target.model, "max_tokens": 256, "stream": False, "messages": [{"role": "user", "content": prompt}]}
    r = requests.post(f"{target.base_url}/v1/messages", headers=headers, json=payload, timeout=30)
    if r.status_code != 200:
        return False, f"HTTP {r.status_code}: {r.text[:200]}"
    data = r.json()
Confidence
80% confidence
Finding
Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

External Transmission

Medium
Category
Data Exfiltration
Content
def _preflight_anthropic(target: Target, prompt: str) -> tuple:
    headers = {"x-api-key": target.api_key, "anthropic-version": "2023-06-01", "content-type": "application/json"}
    payload = {"model": target.model, "max_tokens": 256, "stream": False, "messages": [{"role": "user", "content": prompt}]}
    r = requests.post(f"{target.base_url}/v1/messages", headers=headers, json=payload, timeout=30)
    if r.status_code != 200:
        return False, f"HTTP {r.status_code}: {r.text[:200]}"
    data = r.json()
Confidence
80% confidence
Finding
Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

External Transmission

Medium
Category
Data Exfiltration
Content
def _preflight_openai(target: Target, prompt: str) -> tuple:
    headers = {"Authorization": f"Bearer {target.api_key}", "content-type": "application/json"}
    payload = {"model": target.model, "max_tokens": 32, "stream": False, "messages": [{"role": "user", "content": prompt}]}
    r = requests.post(f"{target.base_url}/chat/completions", headers=headers, json=payload, timeout=20)
    if r.status_code != 200:
        return False, f"HTTP {r.status_code}: {r.text[:200]}"
    data = r.json()
Confidence
80% confidence
Finding
Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

External Transmission

Medium
Category
Data Exfiltration
Content
def _preflight_openai(target: Target, prompt: str) -> tuple:
    headers = {"Authorization": f"Bearer {target.api_key}", "content-type": "application/json"}
    payload = {"model": target.model, "max_tokens": 32, "stream": False, "messages": [{"role": "user", "content": prompt}]}
    r = requests.post(f"{target.base_url}/chat/completions", headers=headers, json=payload, timeout=20)
    if r.status_code != 200:
        return False, f"HTTP {r.status_code}: {r.text[:200]}"
    data = r.json()
Confidence
80% confidence
Finding
Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.