Back to skill

Security audit

Agent Native Design

Security checks for vulnerabilities and agentic risk

Overview

The skill is mostly coherent CLI-design guidance, but its testing guide includes unsafe shell examples that could execute commands supplied by the CLI being tested.

Install only if you treat the testing reference as design guidance, not a script to run blindly. Replace the eval-based harnesses with structured command manifests or argv arrays, and run any CLI review tests in a sandbox with disposable credentials and non-production data.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (2)

T09 · Insecure Skill Coding Practices

Error
Location
references/testing.md:17
Finding

Shell Command Injection Through Dynamic Command Evaluation

Content
View full analysis

Vulnerability Details

File Location: references/testing.md, lines 17–18
Vulnerability Type: Shell command injection through eval
Risk Level: High

Vulnerable Code

bash
# Smoke: every documented command must produce parseable JSON.
for cmd in $(tool _internal list-commands); do
  out=$(eval "$cmd --dry-run" 2>/dev/null) || true
  echo "$out" | jq -e 'has("ok")' > /dev/null \
    || { echo "FAIL: $cmd produced non-JSON or missing 'ok'"; exit 1; }
done

Technical Analysis

The test harness treats output from the CLI under test as trusted shell source code. tool _internal list-commands determines the strings assigned to cmd, and eval reparses each resulting string using the full shell grammar.

If the reviewed CLI is malicious or compromised, it can include shell metacharacters, command substitutions, redirects, pipelines, or additional commands in its output. These constructs will execute with the privileges and environment of the person or CI runner performing the audit.

The command-substitution loop also applies shell word splitting and pathname expansion before eval, making the behavior unsafe and unreliable even for benign command strings containing spaces or wildcard characters.

Attack Path

  1. A developer or CI system applies the documented test harness to an untrusted CLI.
  2. The harness invokes tool _internal list-commands.
  3. The CLI returns attacker-controlled shell syntax as part of a purported command entry.
  4. The loop assigns that output to cmd.
  5. eval "$cmd --dry-run" reparses the value as shell code.
  6. The injected commands execute under the tester's account and inherit the test process's accessible environment and filesystem permissions.

Impact Assessment

Successful exploitation provides arbitrary command execution with the privileges of the user or CI runner executing the example. Depending on the surrounding environment, an attacker could:

  • Read or modify files accessible to the te ...[truncated 457 chars]
Remediation
View remediation

Remediation Suggestions

  • Remove eval and never interpret CLI-produced text as shell source code.
  • Make the internal discovery command return structured data, preferably JSON arrays containing an executable and separate arguments.
  • Parse the output with jq and invoke commands through Bash arrays so each argument remains data rather than syntax.
  • Alternatively, maintain a trusted, test-owned manifest of commands instead of accepting executable command lines from the program under test.
  • Validate discovered executable and subcommand names against strict allowlists.
  • Avoid for cmd in $(...); use a format-preserving loop such as while IFS= read -r for line data, although this alone does not make evaluation safe.
  • Run tests of untrusted CLIs in a sandbox with a minimal environment, read-only source mounts, no production credentials, and denied network access unless explicitly required.

A safer design is for discovery to emit argument arrays, for example:

bash
tool _internal list-commands-json |
  jq -c '.[]' |
  while IFS= read -r entry; do
    mapfile -t argv < <(jq -r '.[]' <<<"$entry")
    out=$("${argv[@]}" --dry-run 2>/dev/null) || true
    jq -e 'has("ok")' <<<"$out" >/dev/null ||
      { echo "FAIL: command produced non-JSON or missing 'ok'" >&2; exit 1; }
  done

The exact implementation should ensure that no value originating from the reviewed CLI is reparsed as shell syntax.

T09 · Insecure Skill Coding Practices

Error
Location
references/testing.md:158
Finding

Shell Command Injection in Mutating-Command Dry-Run Harness

Content
View full analysis

Vulnerability Details

File Location: references/testing.md, lines 158–162
Vulnerability Type: Shell command injection through eval
Risk Level: High

Vulnerable Code

bash
for cmd in $(tool _internal list-mutating-commands); do
  out=$(eval "$cmd --dry-run")
  echo "$out" | jq -e '.dry_run == true and .would_request' > /dev/null \
    || { echo "FAIL: $cmd --dry-run is missing dry_run/would_request"; exit 1; }
done

Technical Analysis

The dry-run safety test evaluates strings emitted by tool _internal list-mutating-commands as shell commands. The CLI under review therefore controls input passed to eval.

eval performs another round of shell parsing. An entry containing command separators, substitutions, redirects, or pipelines can escape the intended --dry-run invocation and execute arbitrary commands. Appending --dry-run does not constrain the preceding string or neutralize shell syntax.

This is particularly problematic because the harness is intended to test potentially state-changing commands. Users may run it with credentials and access to real backing services, increasing the consequences of arbitrary code execution.

Attack Path

  1. A tester runs the documented dry-run safety harness against a malicious or compromised CLI.
  2. The CLI emits a crafted entry from _internal list-mutating-commands.
  3. Shell command substitution and loop expansion process the attacker-controlled output.
  4. eval "$cmd --dry-run" interprets the crafted entry as executable shell syntax.
  5. The payload runs before, after, or instead of the expected dry-run command.
  6. The attacker acts with the tester's local or CI permissions and can use resources exposed to that process.

Impact Assessment

The flaw permits arbitrary command execution within the tester's security context. Potential consequences include:

  • Theft of environment-provided API tokens or CI secrets.
  • Unauthorized local file reads and writes.
  • Execution of actu ...[truncated 356 chars]
Remediation
View remediation

Remediation Suggestions

  • Eliminate eval from the dry-run harness.
  • Represent each command as a structured executable-plus-arguments array rather than a shell command string.
  • Keep the inventory of mutating commands in a trusted test fixture or validate discovered commands against a schema and allowlist.
  • Invoke commands with Bash arrays, such as "${argv[@]}" --dry-run, to prevent metacharacters from becoming syntax.
  • Preserve records without word splitting by consuming JSON or NUL-delimited entries.
  • Isolate tests from production credentials and services. Use disposable credentials, mocked endpoints, read-only mounts, and network restrictions.
  • Ensure the backing-store fingerprint check itself points to a disposable test store and cannot be redirected through CLI-controlled configuration.

A safe replacement should follow this pattern:

bash
while IFS= read -r entry; do
  mapfile -t argv < <(jq -r '.[]' <<<"$entry")
  out=$("${argv[@]}" --dry-run)
  jq -e '.dry_run == true and .would_request' <<<"$out" >/dev/null ||
    { echo "FAIL: dry-run contract missing" >&2; exit 1; }
done < <(tool _internal list-mutating-commands-json | jq -c '.[]')

The producer's schema and the parser must guarantee that each argument is passed literally and is never interpreted as shell source.

Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Findings (12)

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared purpose describes a skill focused on CLI design/review for agent-native usability. The provided code does not implement CLI design, review, refactoring, or API-to-CLI conversion behavior. Instead, it is a maintenance/CI-style metadata validation script for the repository, checking documentation strings, metadata presence, license existence, and version alignment with git tags. This is a materially different primary purpose from the declared description, so it should be flagged as a mismatch.

Content

No source excerpt is available for this finding.

Env Variable Harvesting

High
Category
Data Exfiltration
Confidence
80% confidence
Finding

Code enumerates, copies, or searches environment variables for secrets. Bulk environment access can collect credentials unrelated to the skill's stated purpose.

Content

Scanner excerpt · references/examples.md (reported line 55)May include surrounding context.

bash
# Human / system runs once, out of band (shell profile, systemd unit, supervisor):
healthkit auth login                              # browser OAuth2 flow, stores token in keychain
export HEALTHKIT_TOKEN="$(healthkit auth token)"  # token is injected into the agent's environment

# Agent's own commands — it never invokes `auth login` or `auth token`:

Credential Access

High
Category
Privilege Escalation
Confidence
70% confidence
Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · references/examples.md (reported line 55)May include surrounding context.

bash
# Human / system runs once, out of band (shell profile, systemd unit, supervisor):
healthkit auth login                              # browser OAuth2 flow, stores token in keychain
export HEALTHKIT_TOKEN="$(healthkit auth token)"  # token is injected into the agent's environment

# Agent's own commands — it never invokes `auth login` or `auth token`:

Anti-Refusal Statement

High
Category
Anti-Refusal
Confidence
80% confidence
Finding

Skill instructs the agent to omit warnings, disclaimers, or ethical commentary. Stripping safety caveats hides risk from the user and is a common jailbreak preamble.

Content

Scanner excerpt · references/examples.md (reported line 250)May include surrounding context.

md
purge   Purge all records      ← no warning, no gate

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The skill enables implicit invocation without any visible trigger constraints, exclusions, or scope limits. That can cause the agent to auto-select this skill in broader contexts than intended, which is risky here because the skill influences CLI design, authentication patterns, trust boundaries, and safety-tier behavior that could affect downstream system security decisions.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The file explicitly presents purge Purge all records and even notes no warning, no gate, which is a behavior that could severely affect user data or system integrity. Because this is a markdown file describing potentially destructive behavior, it should include an explicit safety warning about irreversibility and the need for strong safeguards.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

This markdown demonstrates tool danger purge --id abc and its --yes bypass path, which implies destructive deletion. The surrounding text explains non-interactive confirmation behavior but does not explicitly warn readers that the example performs an irreversible or data-affecting action.

Content

No source excerpt is available for this finding.

Skill Enumeration

Medium
Category
Agent Snooping
Confidence
80% confidence
Finding

Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Content

Scanner excerpt · scripts/validate-metadata.sh (reported line 23)May include surrounding context.

sh
assert_contains docs/install_CN.md 'skills/agent-native-design ~/.codex/skills/'
assert_contains docs/install_CN.md 'skills/agent-native-design .codex/skills/'

assert_contains skills/agent-native-design/SKILL.md 'Includes sidecar metadata for OpenClaw, Hermes, pi-mono, and OpenAI Codex'
assert_contains README.md 'includes metadata for the platforms listed below'
assert_contains README_CN.md '并为下列平台提供元数据'

Skill Enumeration

Medium
Category
Agent Snooping
Confidence
80% confidence
Finding

Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Content

Scanner excerpt · scripts/validate-metadata.sh (reported line 27)May include surrounding context.

sh
assert_contains docs/install_CN.md 'skills/agent-native-design ~/.codex/skills/'
assert_contains docs/install_CN.md 'skills/agent-native-design .codex/skills/'

assert_contains skills/agent-native-design/SKILL.md 'Includes sidecar metadata for OpenClaw, Hermes, pi-mono, and OpenAI Codex'
assert_contains README.md 'includes metadata for the platforms listed below'
assert_contains README_CN.md '并为下列平台提供元数据'

Skill Enumeration

Medium
Category
Agent Snooping
Confidence
80% confidence
Finding

Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Content

Scanner excerpt · scripts/validate-metadata.sh (reported line 28)May include surrounding context.

sh
assert_contains docs/install_CN.md 'skills/agent-native-design ~/.codex/skills/'
assert_contains docs/install_CN.md 'skills/agent-native-design .codex/skills/'

assert_contains skills/agent-native-design/SKILL.md 'Includes sidecar metadata for OpenClaw, Hermes, pi-mono, and OpenAI Codex'
assert_contains README.md 'includes metadata for the platforms listed below'
assert_contains README_CN.md '并为下列平台提供元数据'

Skill Enumeration

Medium
Category
Agent Snooping
Confidence
80% confidence
Finding

Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Content

Scanner excerpt · scripts/validate-metadata.sh (reported line 41)May include surrounding context.

sh
assert_contains docs/install_CN.md 'skills/agent-native-design ~/.codex/skills/'
assert_contains docs/install_CN.md 'skills/agent-native-design .codex/skills/'

assert_contains skills/agent-native-design/SKILL.md 'Includes sidecar metadata for OpenClaw, Hermes, pi-mono, and OpenAI Codex'
assert_contains README.md 'includes metadata for the platforms listed below'
assert_contains README_CN.md '并为下列平台提供元数据'

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
81% confidence
Finding

This markdown file demonstrates mytool --token $AGENT_GENERATED_TOKEN delete --id abc123, which is a destructive operation affecting records, but the surrounding text discusses trust boundaries rather than warning about deletion impact or confirmation. Under the markdown variant of SQP-2, descriptions that include behaviors affecting user data should disclose the risk.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.