Back to skill

Security audit

Eval

Security checks for vulnerabilities and agentic risk

Overview

The skill is a plausible evaluation/reporting tool, but it asks the agent to run broad local shell checks, load credential-related context, make authenticated external API calls, and persist feedback logs with weak scoping and consent controls.

Install only if you are comfortable with an eval skill that can inspect broad workspace state, run shell snippets, contact Anthropic and monday.com with local credentials, and create ongoing feedback/evaluation records. Before use, replace `source` with a safe config parser, narrow the trigger phrases, require explicit consent for live credentialed checks, and define where logs are stored and how they are deleted.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Error
Location
SKILL.md:7
Finding
Arbitrary Shell Execution Through Sourced Context File## Vulnerability Details **File Location**: `SKILL.md`, lines 7–10 **Vulnerability Type**: Unsafe execution of a local configuration file **Risk Level**: High ### Vulnerable Code ```bash CONTEXT_FILE="/opt/ocana/openclaw/workspace/skills/eval/.context" [ -f "$CONTEXT_FILE" ] && source "$CONTEXT_FILE" # Then use: $OWNER_PHONE, $WORKSPACE, $TASKS_FILE, $MONDAY_TOKEN_FILE, $GOG_CREDS, etc. ``` ### Technical Analysis The shell `source` command interprets the entire `.context` file as executable shell code rather than parsing it as inert configuration data. Consequently, the file may contain arbitrary commands, command substitutions, redirections, function definitions, or environment modifications. If an attacker or compromised local process can modify this context file, triggering the Skill causes the injected commands to execute with the same operating-system privileges and environment as the Agent. The instruction also loads potentially sensitive values—including an owner phone number, API-token locations, and credential references—without establishing that every value is required for the evaluation. This unnecessarily increases secret exposure and the impact of injected code. The issue is conditional on an attacker obtaining write access to `.context` or its parent directory. The reviewed Skill does not itself grant that access. ### Attack Path 1. An attacker or compromised local process obtains write access to `/opt/ocana/openclaw/workspace/skills/eval/.context`. 2. The attacker inserts a shell payload into the file, such as a command that reads credentials, modifies workspace files, or sends collected data over the network. 3. The owner invokes the evaluation Skill. 4. The Agent follows the initialization instruction and executes `source "$CONTEXT_FILE"`. 5. The shell interprets the attacker's payload under the Agent's identity and permissions. 6. The payload can access files, credentials, environment variables, and network resources available to the A ...[truncated 809 chars]
Remediation
## Remediation Suggestions 1. **Do not execute the context file.** Replace `source` with a parser that treats the file strictly as data. 2. **Use an explicit key allowlist.** Accept only configuration keys required by the current operation, such as `WORKSPACE` or `TASKS_FILE`, and reject unexpected entries. 3. **Reject shell syntax.** Disallow command substitutions, backticks, redirections, control operators, function definitions, multiline values, and executable statements. 4. **Load secrets only when required.** Do not import phone numbers, API-token paths, or credential references for checks that do not need them. 5. **Validate file security before reading.** Confirm that the file is a regular file, is owned by the expected account, is not a symbolic link, and is not writable by group or other users. 6. **Apply least privilege.** Run the Skill under an account with access only to the files and network destinations needed for evaluation. 7. **Prefer structured configuration.** Store non-secret settings in JSON or another format parsed by a non-executing parser. Retrieve secrets from a dedicated secret manager at the point of use. 8. **Document the trust boundary.** Explicitly state who may modify the context file and ensure its parent directories have restrictive permissions.
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (11)

Context-Inappropriate Capability

Medium
Confidence
96% confidence
Finding
The skill instructs the agent to source a local context file that may populate numerous sensitive variables such as phone numbers, workspace paths, and credential file locations. For an evaluation/reporting skill, broad secret loading is unnecessary and increases the chance of unintended credential access, misuse, or later exfiltration through other commands or outputs.

Vague Triggers

Medium
Confidence
95% confidence
Finding
Broad trigger phrases like 'check everything' and 'what's working and what isn't' can cause the skill to activate during ordinary conversation. In this skill's context, activation is more dangerous because running eval leads to file inspection, health checks, and possible external API calls, so accidental invocation can produce unintended data access and network activity.

Context-Inappropriate Capability

Medium
Confidence
93% confidence
Finding
The skill performs live third-party API checks using locally stored credentials as part of a general evaluation flow. This expands the skill from passive reporting into active networked operations, creating unnecessary exposure of secrets and metadata and increasing the attack surface if the skill is triggered unexpectedly or abused.

External Transmission

Medium
Category
Data Exfiltration
Content
```bash
# Test Anthropic billing
API_STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
  -H "x-api-key: ${ANTHROPIC_API_KEY:-none}" \
  -H "anthropic-version: 2023-06-01" \
  https://api.anthropic.com/v1/models 2>/dev/null)
Confidence
94% confidence
Finding
This command makes an outbound request to Anthropic using an API key from the environment to test billing status. Even though it only checks an HTTP status code, it still transmits a live credential to an external service and creates unnecessary network side effects for a reporting skill.

External Transmission

Medium
Category
Data Exfiltration
Content
API_STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
  -H "x-api-key: ${ANTHROPIC_API_KEY:-none}" \
  -H "anthropic-version: 2023-06-01" \
  https://api.anthropic.com/v1/models 2>/dev/null)

# Interpret result
if [ "$API_STATUS" = "200" ]; then echo "Billing: ✅ OK"
Confidence
90% confidence
Finding
The skill explicitly targets an external Anthropic endpoint as part of evaluation. In context, this matters because the skill is supposed to summarize system state, not initiate outside connections, and the broad triggers increase the chance of unintended external communication.

External Transmission

Medium
Category
Data Exfiltration
Content
# Test monday.com
if [ -f "$HOME/.credentials/monday-api-token.txt" ]; then
  MONDAY_STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
    -X POST https://api.monday.com/v2 \
    -H "Authorization: $(cat $HOME/.credentials/monday-api-token.txt)" \
    -H "Content-Type: application/json" \
    -d '{"query": "{ me { id } }"}' 2>/dev/null)
Confidence
95% confidence
Finding
The monday.com health check posts to an external API using a token read directly from a local credentials file. This is risky because a generally triggered evaluation skill should not automatically read secrets and send authenticated requests to third parties, especially when a passive status report would suffice.

Missing User Warnings

Medium
Confidence
92% confidence
Finding
The skill describes automatic logging and file writes without warning or asking the user. Even though this specific file lacks implementation for some of those actions, the design intent is to persist user-related data silently, which is unsafe and normalizes covert state changes in response to routine interaction.

Natural-Language Policy Violations

Medium
Confidence
95% confidence
Finding
The skill mandates automatic logging of owner feedback, corrections, and behavior signals without user choice. This can capture sensitive conversational content and create behavioral profiles, especially because corrections and repeated questions may contain personal or confidential information.

Ssd 3

Medium
Confidence
93% confidence
Finding
Automatic retention of owner feedback, corrections, and evaluation records risks collecting user-provided content without minimization or clear boundaries. In an assistant-management skill, these logs may aggregate operational history, mistakes, and potentially sensitive task details, increasing privacy and insider-risk exposure over time.

Intent-Code Divergence

Low
Confidence
95% confidence
Finding
This section makes an active behavioral claim: feedback signals are to be detected and logged automatically, immediately, and not batched. In the actual file, there is no code or command sequence implementing signal detection or persistence, so the documentation overstates what the skill actually does.

Intent-Code Divergence

Low
Confidence
97% confidence
Finding
The 'Weekly Eval File' section presents saving an evaluation artifact as part of the skill's behavior. Yet the file only shows read/check commands and report formatting instructions, with no write operation creating the stated output path, so the documented intent diverges from the implemented behavior.

Static analysis

No suspicious patterns detected.