Back to skill

Security audit

OATDA Compare Models

Security checks for vulnerabilities and agentic risk

Overview

This skill does what it claims, but its shell example can turn a model-comparison prompt into executable shell syntax, which users should review before installing.

Review this skill before installing. It is not trying to hide persistence or unrelated access, but the documented curl pattern should be replaced with a safer structured JSON construction such as jq --arg or a scoped HTTP client before using arbitrary prompts, especially prompts copied from untrusted sources. Avoid sending secrets, personal data, proprietary documents, or credentials through model-comparison requests unless that sharing is intended.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Error
Location
SKILL.md:59
Finding

Shell Command Injection Through Unsafe Prompt Substitution

Content
View full analysis
/dev/null | jq -r '.profiles[.defaultProfile].apiKey' 2>/dev/null)}" && \ curl -s -X POST "https://oatda.com/api/v1/compare" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OATDA_API_KEY" \ -d '{ "prompt": "", "models": [ {"provider": "openai", "modelId": "gpt-5"}, {"provider": "anthropic", "modelId": "claude-sonnet-4-5-20250929"}, {"provider": "google", "modelId": "gemini-3-pro-preview"} ], "temperature": 0.7, "maxTokens": 1024, "stream": false }' ``` The subsequent instruction states: ```text Replace `` with the user's prompt (JSON-escape special characters). ``` ### Technical Analysis The Skill instructs the Agent to replace `` inside a shell command whose HTTP request body is enclosed in shell single quotes. Although the documentation tells the Agent to JSON-escape special characters, JSON escaping does not protect a value embedded directly into shell source. In particular, a single quote in an attacker-controlled prompt can terminate the shell's single-quoted `-d` argument. Text following that quote can then be interpreted as shell syntax rather than JSON data. Shell metacharacters such as semicolons, redirections, command substitutions, and comment markers may consequently cause additional commands to execute. The underlying security failure is the use of textual substitution to combine untrusted data and executable shell syntax. JSON encoding and shell argument encoding are distinct operations; satisfying the former does not satisfy the latter. ### Attack Path 1. An attacker asks the Agent to compare models using a prompt containing a single ...[truncated 1674 chars]
Remediation
View remediation
/dev/null)}" payload=$( jq -n --arg prompt "$USER_PROMPT" '{ prompt: $prompt, models: [ {provider: "openai", modelId: "gpt-5"}, {provider: "anthropic", modelId: "claude-sonnet-4-5-20250929"}, {provider: "google", modelId: "gemini-3-pro-preview"} ], temperature: 0.7, maxTokens: 1024, stream: false }' ) curl --silent --show-error --fail-with-body \ -X POST "https://oatda.com/api/v1/compare" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OATDA_API_KEY" \ --data-binary "$payload" ``` Additional hardening measures: 1. Obtain the prompt through a variable or structured tool parameter rather than generating shell code containing it. 2. Prefer a structured HTTP client or Agent HTTP tool that accepts headers and JSON values separately. 3. Never use `eval`, command substitution, or dynamically generated shell scripts to process the prompt. 4. Validate that the API key is neither empty nor the literal value `null` before issuing a request. 5. Run the Skill in a sandbox with minimum filesystem and network permissions. 6. Restrict access to `~/.oatda/credentials.json` to the owning user, such as mode `0600`. 7. Avoid displaying even partial API-key prefixes unless operationally necessary, as prefixes may still constitute sensitive metadata. 8. Add regression tests using prompts containing single quotes, newlines, command substitutions, semicolons, redirections, and other shell metacharacters. ]]>
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
Findings (8)

Credential Access

High
Category
Privilege Escalation
Confidence
70% confidence
Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · SKILL.md (reported line 10)May include surrounding context.

md
"openclaw":
      {
        "emoji": "⚔️",
        "requires": { "bins": ["curl", "jq"], "env": ["OATDA_API_KEY"], "config": ["~/.oatda/credentials.json"] },
        "primaryEnv": "OATDA_API_KEY",
      },
  }

Credential Access

High
Category
Privilege Escalation
Confidence
88% confidence
Finding

The skill instructs inline reading of an API key from a local credentials file and exporting it into the shell environment for subsequent commands. While meant for legitimate authentication, this is security-sensitive because it increases exposure of credentials to shell history, subprocesses, debugging output, or accidental reuse by unrelated commands in the same session.

Content

Scanner excerpt · SKILL.md (reported line 25)May include surrounding context.

All commands need the OATDA API key. Resolve it inline for each exec call:

bash
export OATDA_API_KEY="${OATDA_API_KEY:-$(cat ~/.oatda/credentials.json 2>/dev/null | jq -r '.profiles[.defaultProfile].apiKey' 2>/dev/null)}"

If the key is empty or null, tell the user to get one at https://oatda.com and configure it.

Credential Access

High
Category
Privilege Escalation
Confidence
90% confidence
Finding

This command both loads a local API key from disk and immediately uses it in an outbound request, creating a compound credential-exposure path. Although the text says not to print the full key, the pattern still increases the blast radius if shell logging, process inspection, or command reuse occurs.

Content

Scanner excerpt · SKILL.md (reported line 59)May include surrounding context.

Wichtig: Feld heißt modelId (nicht model!), models ist Array-of-objects, max 8.

bash
export OATDA_API_KEY="${OATDA_API_KEY:-$(cat ~/.oatda/credentials.json 2>/dev/null | jq -r '.profiles[.defaultProfile].apiKey' 2>/dev/null)}" && \
curl -s -X POST "https://oatda.com/api/v1/compare" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OATDA_API_KEY" \

Credential Access

High
Category
Privilege Escalation
Confidence
89% confidence
Finding

The example normalizes a workflow in which a local credential is read from disk and used in a network request, which is a real credential-handling concern even if not overtly malicious. Repetition in examples makes insecure handling more likely to be copied verbatim by downstream users or agents.

Content

Scanner excerpt · SKILL.md (reported line 141)May include surrounding context.

User: "Compare how GPT-5, Claude Sonnet 4.5, and Gemini 3 Pro explain recursion in one sentence."

bash
export OATDA_API_KEY="${OATDA_API_KEY:-$(cat ~/.oatda/credentials.json 2>/dev/null | jq -r '.profiles[.defaultProfile].apiKey' 2>/dev/null)}" && \
curl -s -X POST "https://oatda.com/api/v1/compare" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OATDA_API_KEY" \

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The trigger description includes broad conditions such as 'pick the best LLM for a task' and 'find the cheapest/best/fastest answer,' which could match many ordinary user requests rather than a clearly scoped invocation. The file does not provide exclusion conditions or negative examples to clarify when the skill should not activate.

Content

No source excerpt is available for this finding.

External Transmission

Medium
Category
Data Exfiltration
Confidence
92% confidence
Finding

This skill instructs sending user-provided prompts and model selections to an external service over the network. Even though external API use is the skill's purpose, it still creates a real data-exposure boundary: sensitive prompts, proprietary text, or regulated data could be transmitted to a third party if used without explicit consent and data-handling safeguards.

Content

Scanner excerpt · SKILL.md (reported line 60)May include surrounding context.

bash
export OATDA_API_KEY="${OATDA_API_KEY:-$(cat ~/.oatda/credentials.json 2>/dev/null | jq -r '.profiles[.defaultProfile].apiKey' 2>/dev/null)}" && \
curl -s -X POST "https://oatda.com/api/v1/compare" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OATDA_API_KEY" \
  -d '{

External Transmission

Medium
Category
Data Exfiltration
Confidence
91% confidence
Finding

The example repeats the pattern of transmitting prompt contents and model requests to an external API, reinforcing a workflow that sends user data off-platform. In a comparison skill, this is expected behavior, but it remains security-relevant because it may expose confidential inputs to multiple providers simultaneously.

Content

Scanner excerpt · SKILL.md (reported line 142)May include surrounding context.

bash
export OATDA_API_KEY="${OATDA_API_KEY:-$(cat ~/.oatda/credentials.json 2>/dev/null | jq -r '.profiles[.defaultProfile].apiKey' 2>/dev/null)}" && \
curl -s -X POST "https://oatda.com/api/v1/compare" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OATDA_API_KEY" \
  -d '{

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
95% confidence
Finding

The line begins with 'Wichtig,' introducing an instruction in German inside otherwise English documentation. This imposes a language/locale inconsistency without user opt-in or a documented reason for the locale constraint.

Content

No source excerpt is available for this finding.