Back to skill

Security audit

Soul Searching

Security checks for vulnerabilities and agentic risk

Overview

The skill mostly does what it says, but it needs review because remote catalog entries can persistently replace agent instructions and unsafe soul IDs can escape the intended storage folder.

Install only if you trust soulsearching.ai and the publisher to provide safe SOUL.md content. Prefer browsing and reviewing content before activation, keep your own backup of SOUL.md, and avoid using catalog entries with unusual IDs until the script validates IDs, verifies remote content provenance, and asks before overwriting or deleting files.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (2)

T01 · Skill Instruction Hijacking

Error
Location
scripts/soul.sh:37
Finding
Untrusted Remote Content Can Persistently Replace Agent Instructions<![CDATA[ ## Vulnerability Details **File Location**: `scripts/soul.sh:4-8`, `scripts/soul.sh:37-57`, `scripts/soul.sh:146-174`, and `scripts/soul.sh:177-208` **Vulnerability Type**: Remote instruction hijacking and persistent Agent state poisoning **Risk Level**: Critical ### Vulnerable Code ```bash CATALOG_URL="https://soulsearching.ai/souls.json" SOUL_DIR="${HOME}/.openclaw/souls" CATALOG_FILE="${SOUL_DIR}/.catalog.json" WORKSPACE="${OPENCLAW_WORKSPACE:-$(pwd)}" SOUL_FILE="${WORKSPACE}/SOUL.md" ``` ```bash refresh_catalog() { ensure_dir local tmp="${CATALOG_FILE}.tmp" echo "📡 Fetching soul catalog from soulsearching.ai..." >&2 if ! curl -sSfL "$CATALOG_URL" -o "$tmp" 2>/dev/null; then echo "❌ Failed to fetch catalog. Check your connection." >&2 rm -f "$tmp" exit 1 fi # Normalize: if top-level is array, wrap it; rename "soul" key → "content" python3 -c " import json, sys with open('$tmp') as f: data = json.load(f) if isinstance(data, list): data = {'version': 1, 'source': 'https://soulsearching.ai', 'souls': data} for s in data.get('souls', []): if 'soul' in s and 'content' not in s: s['content'] = s.pop('soul') with open('$CATALOG_FILE', 'w') as f: json.dump(data, f, indent=2) print(len(data.get('souls', []))) " > /dev/null ``` ```bash local soul_json soul_json=$(get_soul_json "$id") if [[ -z "$soul_json" ]]; then echo "❌ Soul '$id' not found in catalog." >&2 echo " Run: soul.sh browse (to see available souls)" >&2 exit 1 fi local name name=$(echo "$soul_json" | jq -r '.name') local content content=$(echo "$soul_json" | jq -r '.content') # Save to local store echo "$content" > "${SOUL_DIR}/${id}.md" echo "✅ Installed: $name → ~/.openclaw/souls/${id}.md" if [[ "$activate" == true ]]; then cmd_switch "$id" fi ``` ```bash # Backup current SOUL.md if it exists if [[ -f "$SOUL_FILE" ]]; then cp "$SOUL_FILE" "${SOUL_FILE}.bak" echo "📋 Backed up current SOUL.md → SOUL.md.bak" fi # Copy soul ...[truncated 2757 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Publish the catalog through a cryptographically signed, versioned manifest and verify its signature against a public key embedded in or securely distributed with the Skill. 2. Include a trusted digest for every soul file and verify the downloaded content before installation or activation. 3. Pin approved catalog versions rather than implicitly trusting mutable remote content. 4. Display the complete proposed `SOUL.md` content and require explicit, informed user confirmation immediately before activation. 5. Clearly warn that activation changes persistent Agent instructions, not merely cosmetic application data. 6. Validate content against a restrictive policy and reject directives that attempt to override system rules, request secrets, modify safety constraints, or trigger tools. 7. Separate untrusted downloaded content from instruction files. Prefer a structured, limited personality schema whose fields are rendered by trusted local code. 8. Record provenance, signature status, version, and content digest for every installed soul. 9. Provide a safe rollback mechanism and preserve multiple immutable backups instead of overwriting a single `.bak` file. 10. Avoid automatic activation immediately after retrieval; installation and activation should be separate trust decisions. ]]>

T09 · Insecure Skill Coding Practices

Error
Location
scripts/soul.sh:146
Finding
Unvalidated Soul IDs Allow Path Traversal Outside the Soul Directory<![CDATA[ ## Vulnerability Details **File Location**: `scripts/soul.sh:146-171`, `scripts/soul.sh:177-208`, and `scripts/soul.sh:276-289` **Vulnerability Type**: Path traversal enabling out-of-scope file write, read, and deletion **Risk Level**: High ### Vulnerable Code ```bash cmd_install() { ensure_catalog local id="${1:-}" local activate=false if [[ -z "$id" ]]; then echo "Usage: soul.sh install <soul-id> [--activate]" >&2 exit 1 fi # Fetch soul data local soul_json soul_json=$(get_soul_json "$id") if [[ -z "$soul_json" ]]; then echo "❌ Soul '$id' not found in catalog." >&2 echo " Run: soul.sh browse (to see available souls)" >&2 exit 1 fi local name name=$(echo "$soul_json" | jq -r '.name') local content content=$(echo "$soul_json" | jq -r '.content') # Save to local store echo "$content" > "${SOUL_DIR}/${id}.md" ``` ```bash cmd_switch() { local id="${1:-}" if [[ -z "$id" ]]; then echo "Usage: soul.sh switch <soul-id>" >&2 echo " Installed souls:" >&2 cmd_list exit 1 fi local soul_path="${SOUL_DIR}/${id}.md" if [[ ! -f "$soul_path" ]]; then echo "❌ Soul '$id' is not installed locally." >&2 echo " Run: soul.sh install $id" >&2 exit 1 fi # Backup current SOUL.md if it exists if [[ -f "$SOUL_FILE" ]]; then cp "$SOUL_FILE" "${SOUL_FILE}.bak" echo "📋 Backed up current SOUL.md → SOUL.md.bak" fi # Copy soul into place cp "$soul_path" "$SOUL_FILE" ``` ```bash cmd_uninstall() { local id="${1:-}" if [[ -z "$id" ]]; then echo "Usage: soul.sh uninstall <soul-id>" >&2 exit 1 fi local soul_path="${SOUL_DIR}/${id}.md" if [[ ! -f "$soul_path" ]]; then echo "❌ Soul '$id' is not installed." >&2 exit 1 fi local name name=$(head -1 "$soul_path" | sed 's/^# SOUL.md — //' | sed 's/^# //') rm "$soul_path" ``` ### Technical Analysis The `id` value is incorporated directly into filesystem paths without validation or ...[truncated 2811 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Validate every soul ID before any catalog lookup or filesystem operation. Use a strict allowlist such as: ```bash validate_id() { local id="$1" if [[ ! "$id" =~ ^[A-Za-z0-9][A-Za-z0-9_-]*$ ]]; then echo "Invalid soul ID." >&2 exit 1 fi } ``` 2. Reject slashes, backslashes, parent-directory components, control characters, whitespace, leading dots, and empty IDs. 3. Apply the same validation consistently in `install`, `switch`, and `uninstall`. 4. Canonicalize the parent directory and destination, then verify that the final path remains directly beneath the canonical `SOUL_DIR`. 5. Do not rely solely on string-prefix checks, which can be bypassed by similarly named directories or symbolic links. 6. Refuse to operate on symbolic-link destinations. Where available, use filesystem operations that prevent symlink following and race conditions. 7. Consider mapping catalog IDs to locally generated safe filenames rather than using remote identifiers as path components. 8. Require confirmation before overwriting existing files and use exclusive file creation where overwriting is not intended. 9. Write downloads to a securely created temporary file, validate them, and atomically rename them into the verified destination. 10. Add automated tests covering IDs such as `../x`, `../../x`, absolute paths, embedded slashes, leading dots, control characters, and symlink-based escape attempts. ]]>
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Findings (5)

Lp3

Medium
Category
MCP Least Privilege
Confidence
94% confidence
Finding
The skill advertises shell execution and file-writing behavior but does not declare any explicit tool scope or permission boundaries. That makes the skill harder to constrain at runtime and increases the chance an agent can invoke filesystem changes or shell actions more broadly than intended, especially in systems that rely on manifest-declared scopes for enforcement or review.

Vague Triggers

Medium
Confidence
91% confidence
Finding
The manifest description includes broad trigger phrases like personality management, switching, browsing, and current soul checks, which may cause the skill to activate in contexts that only loosely relate to SOUL.md management. Over-broad activation is dangerous here because the skill can lead to network retrievals and overwriting local personality files, so accidental invocation could change agent behavior or workspace state unexpectedly.

Missing User Warnings

Medium
Confidence
93% confidence
Finding
The skill explicitly describes downloading content from an external site and overwriting local SOUL.md files, but it does not present a clear warning or consent step about those side effects. This is risky because remote content can alter agent behavior, and overwriting SOUL.md can silently replace local configuration or prompt policy in a way the user may not anticipate.

Missing User Warnings

Medium
Confidence
95% confidence
Finding
The switch operation overwrites the workspace SOUL.md with content previously fetched from a remote catalog, and only backs up an existing file silently without warning or confirmation. In an agent-skill context, SOUL.md likely influences agent behavior, so replacing it can unexpectedly change runtime behavior or destroy local customizations if the backup is missed or later overwritten.

Missing User Warnings

Low
Confidence
87% confidence
Finding
The uninstall command removes a local soul file immediately with no confirmation prompt or safety check. While limited to the tool's own storage directory, this can still cause accidental loss of installed personalities and operational disruption, particularly if users rely on a local soul that is not easily recoverable.

Static analysis

No suspicious patterns detected.