ClawHub
Back to skill

Security audit

Find Souls

Security checks for vulnerabilities and agentic risk

Overview

This skill does what it says: it fetches persona prompts and installs them into a project SOUL.md, with backups and user confirmation before overwriting.

Install only if you trust agent-souls.com as a source of persona prompts. Before resetting the conversation, review the downloaded SOUL.md for instructions that ask for secrets, override safety expectations, or request unrelated tool access. Keep the .soul_backups directory if you may want to roll back, and clear ~/.cache/agent-souls/search.json if you want to force a fresh index.

A.I.G

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Error
Location
SKILL.md:49
Finding

Unverified Remote Persona Content Installed as Trusted Agent Instructions

Content
View full analysis

Vulnerability Details

File Location: SKILL.md, lines 49-68
Vulnerability Type: Unvalidated remote content installation
Risk Level: High

Vulnerable Code

md
Construct the download URL:

https://agent-souls.com{url}SOUL.md # Chinese https://agent-souls.com{url}SOUL.en.md # English

text

Where `{url}` is the `url` field from search.json (e.g. `/real_world/confucius/`).

Use `WebFetch` to download the raw content of the chosen SOUL file.

### Step 4: Backup & Install

Before replacing, always back up the existing SOUL.md:

1. Check if `SOUL.md` exists in the current working directory.
2. If it exists, create `.soul_backups/` directory if it doesn't exist, then:
   - **First time only**: if `.soul_backups/SOUL.md.original` does **not** exist, copy SOUL.md to `.soul_backups/SOUL.md.original`. This preserves the user's original persona and is never overwritten.
   - **Every time**: copy SOUL.md to `.soul_backups/SOUL.md.<slug>` where `<slug>` is the soul's unique identifier extracted from the `url` field.
3. Write the downloaded content to `SOUL.md` in the current working directory.

Technical Analysis

The skill obtains a path from the remotely maintained search.json index, constructs a download URL from that value, downloads the resulting document, and writes it directly to SOUL.md. The installed file is intended to become trusted agent persona instructions after the user resets the conversation.

The workflow does not require:

  • Cryptographic signature or checksum verification.
  • Pinning to a reviewed, immutable version.
  • Validation that the index path has an allowed format.
  • Enforcement of the expected origin after URL resolution or redirects.
  • Content-size or structure limits.
  • Inspection for prompt injection, safety-policy overrides, or sensitive tool requests.
  • A preview of the complete document before installation.

HTTPS protects ...[truncated 1847 chars]

Remediation
View remediation

Remediation Suggestions

  1. Distribute persona documents through signed, immutable releases and verify a trusted signature before installation.
  2. Pin each approved persona version to a cryptographic digest and reject content whose digest does not match.
  3. Validate the index url against a strict relative-path allowlist, such as an expected category and slug pattern. Reject absolute URLs, traversal sequences, query manipulation, fragments, encoded separators, and unexpected characters.
  4. Enforce HTTPS and the exact agent-souls.com origin both before and after redirects. Prefer disabling redirects.
  5. Apply document size limits and validate that the response is a plain Markdown persona document.
  6. Scan downloaded content for instruction overrides, requests for secrets, unauthorized tool use, external data transmission, and attempts to modify safety constraints.
  7. Display the complete downloaded content, source URL, version, and verified digest to the user before installation.
  8. Require explicit confirmation after the preview and immediately before replacing SOUL.md.
  9. Install first to a temporary file in the same directory, validate it, and then perform an atomic replacement.
  10. Retain the existing backup and rollback controls, but clearly state that backups provide recovery rather than authenticity or prevention.

SkillSpector

Vulnerability Patterns
  • Memory PoisoningPersistent Context Injection, Context Window Stuffing, Memory Manipulation
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (6)

Memory Manipulation

High
Category
Memory Poisoning
Confidence
80% confidence
Finding

Skill manipulates agent memory, state, or stored context. Memory corruption can alter personality, override safety rules, or cause unpredictable behavior.

Content

Scanner excerpt · README.md (reported line 38)May include surrounding context.

md
4. Back up your current SOUL.md (if any)
5. Install the new SOUL.md

After installation, reset your conversation to load the new persona.

### Rollback to a previous soul

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The README explicitly says the skill will download and install a SOUL.md into the user's project and back up or replace any existing SOUL.md, but it does not clearly warn that local project files will be modified. Because persona prompt files can materially change agent behavior, silent replacement of project prompt state can surprise users and create integrity and workflow risks even if the feature is intentional.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The manifest description says to use the skill when the user wants to roleplay as a historical figure, fictional character, or expert persona, and to browse AI character prompts. These are broad natural-language conditions without clear trigger boundaries or exclusions, which can cause unintended invocation for ordinary discussion about personas or characters.

Content

No source excerpt is available for this finding.

Session Persistence

Medium
Category
Rogue Agent
Confidence
91% confidence
Finding

The skill persists remotely fetched search metadata in a user-scoped cache under ~/.cache/agent-souls/search.json, creating session persistence across runs. While the cached data is only an index, persisting untrusted remote content can influence future behavior, become stale or poisoned, and create a cross-session trust channel if the upstream source is compromised or manipulated.

Content

Scanner excerpt · SKILL.md (reported line 23)May include surrounding context.

md
2. If the cache is **fresh** (< 1 day old): read the local file directly with the `Read` tool.
3. If the cache is **stale** (>= 1 day old) or **missing**:
   - `WebFetch https://agent-souls.com/search.json` to download it.
   - Create directory `~/.cache/agent-souls/` if it doesn't exist.
   - Write the downloaded JSON to `~/.cache/agent-souls/search.json`.

The JSON is an array of objects with these fields:

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
85% confidence
Finding

The README identifies a remote source and search index on agent-souls.com, but does not warn users that the skill fetches remote content that may reveal queries or import untrusted prompt content into the project. In a prompt-installation skill, remote retrieval is security-relevant because downloaded persona files directly influence later model behavior.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
95% confidence
Finding

The skill allows explicit language choice, but otherwise forces SOUL.en.md whenever the conversation is in English or unknown. Defaulting to a specific language for unknown-language situations is a locale policy issue because it does not first ask the user for their preference.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.