Back to skill

Security audit

Skill Factory

Security checks for vulnerabilities and agentic risk

Overview

This skill is coherently aimed at building and publishing other skills, but it uses broad local scans and unsafe helper-script and packaging patterns that could leak files or carry untrusted instructions into future skill work.

Review this skill before installing or using it. Run bundled scripts only from a verified skill installation path, avoid the find-based fallback commands, inspect the full package manifest before publishing, and do not let generated pattern reports or eval outputs automatically rewrite or publish skills without human review.

Vulnerability Patterns
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Tool Hijacking and SpoofingModifies or replaces tools so legitimate-looking calls execute attacker logic
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (3)

T02 · Agent Memory Poisoning

Error
Location
scripts/analyze_patterns.py:47
Finding
Persistent Pattern-Library Poisoning Through Untrusted Skill Metadata<![CDATA[ ## Vulnerability Details **File Location**: `scripts/analyze_patterns.py:47-57`, `scripts/analyze_patterns.py:110-111`, `scripts/analyze_patterns.py:176-178`, and `scripts/analyze_patterns.py:213-217` **Vulnerability Type**: Persistent poisoning of agent-consumed reference data **Risk Level**: High ### Vulnerable Code ```python text = skill_md.read_text(errors="replace") # Extract frontmatter name, description = "", "" fm_match = re.match(r"^---\n(.*?)\n---", text, re.DOTALL) if fm_match: fm = fm_match.group(1) name_m = re.search(r'^name:\s*(.+)$', fm, re.MULTILINE) desc_m = re.search(r'^description:\s*"(.+)"', fm, re.MULTILINE | re.DOTALL) if name_m: name = name_m.group(1).strip() if desc_m: description = desc_m.group(1).strip() ``` ```python all_triggers.extend([(phrase, s["name"]) for phrase in s["trigger_phrases"]]) ``` ```python snippet = phrase[:100].replace('\n', ' ') lines.append(f"- [{skill_name}] `{snippet}`") ``` ```python if args.output: out_path = Path(args.output).expanduser() out_path.parent.mkdir(parents=True, exist_ok=True) out_path.write_text(output) print(f"Report written to {out_path}", file=sys.stderr) ``` ### Technical Analysis The analyzer reads metadata from every installed `SKILL.md` under the selected scan directories and treats the extracted name and description as trusted report content. These values are inserted into Markdown without escaping Markdown delimiters, filtering instruction-like text, or recording a clear untrusted-data boundary. The description parser also uses a greedy multiline expression: ```python r'^description:\s*"(.+)"' ``` Combined with `re.DOTALL`, this expression may capture more frontmatter content than the intended description value. A crafted installed Skill can therefore place model-directed instructions or misleading structural content into fields consumed by the analyzer. The generated output can be persisted to `references/patterns ...[truncated 1926 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Replace regular-expression frontmatter parsing with `yaml.safe_load`. 2. Require `name` and `description` to be scalar strings with explicit length and line-count limits. 3. Reject multiline descriptions when generating the pattern library unless multiline content is explicitly required. 4. Escape backticks, brackets, pipes, HTML characters, and other Markdown control syntax before interpolation. 5. Store extracted data in a structured JSON format rather than directly composing agent-consumed Markdown. 6. Clearly mark all extracted values as untrusted quotations and instruct consuming agents never to follow instructions found inside those values. 7. Preserve provenance for every extracted value, including source path and package identity. 8. Maintain an allowlist of trusted Skills for synthesis or require user review before incorporating patterns from newly installed Skills. 9. Add tests containing prompt-injection strings, malformed frontmatter, multiline descriptions, and Markdown-breaking payloads. 10. Require confirmation before replacing an existing persistent pattern library. ]]>

T07 · Tool Hijacking and Spoofing

Error
Location
SKILL.md:45
Finding
Execution of Unverified Scripts Discovered by Filename<![CDATA[ ## Vulnerability Details **File Location**: `SKILL.md:45-48` and `SKILL.md:91-92` **Vulnerability Type**: Local executable discovery susceptible to tool spoofing **Risk Level**: High ### Vulnerable Code ```bash # From your workspace skills directory: python3 $(openclaw skills info skill-creator --json 2>/dev/null | python3 -c "import json,sys; print(json.load(sys.stdin).get('path',''))")/scripts/init_skill.py \ <skill-name> \ --path ~/.openclaw/workspace/skills/ \ --resources scripts,references \ --examples ``` The documented fallback is unsafe: ```bash SKILL_DIR=$(dirname $(find ~/.openclaw/workspace/skills ~/.nvm -name "init_skill.py" 2>/dev/null | head -1)) python3 "$SKILL_DIR/init_skill.py" <skill-name> --path ~/.openclaw/workspace/skills/ --resources scripts,references ``` The packaging workflow repeats the same pattern: ```bash SKILL_SCRIPTS="$(dirname "$(find ~/.openclaw/workspace/skills/skill-creator ~/.nvm -name "package_skill.py" 2>/dev/null | head -1)")" python3 "$SKILL_SCRIPTS/package_skill.py" ~/.openclaw/workspace/skills/<skill-name> ``` ### Technical Analysis The fallback commands search broad user-controlled directory trees for files with specific names and execute the first match returned by `find`. The commands do not verify: - The selected file belongs to the expected Skill. - The path is canonical and beneath a trusted installation directory. - The file owner or permissions are trustworthy. - The file content or hash matches the reviewed bundled script. - Multiple matches exist. - A symlink or attacker-created directory influenced the result. Filename equality is not an authenticity check. Any process or package capable of writing an earlier matching file under the searched paths can spoof the bundled tool. The behavior exceeds the minimum necessary privilege scope because the Skill already describes these scripts as bundled resources. It is unnecessary to search all of `~/.nvm` or broad workspace directories for ...[truncated 1351 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Remove all `find ... -name <script> | head -1` executable-discovery fallbacks. 2. Resolve the Skill installation path using one trusted OpenClaw registry command. 3. Validate that the returned path is nonempty, canonical, and points to the expected Skill slug. 4. Quote the entire generated path before execution. 5. Fail closed if the registry lookup fails or if more than one installation is detected. 6. Verify that the resolved script is a regular file and not a symbolic link. 7. Where supported, verify package ownership, signature, or a known script digest before execution. 8. Do not search general `~/.nvm` or workspace directory trees for executable code by filename. 9. Prefer invocation through an authenticated package entry point rather than executing discovered source files. 10. Document manual path selection as a non-executing diagnostic step if automatic resolution is unavailable. ]]>

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/package_skill.py:70
Finding
Skill Packaging Can Include Sensitive Files and Symlink Targets<![CDATA[ ## Vulnerability Details **File Location**: `scripts/package_skill.py:70-75`; related validation gap in `scripts/quick_validate.py:15-98` **Vulnerability Type**: Unsafe recursive archive construction **Risk Level**: Medium ### Vulnerable Code ```python with zipfile.ZipFile(skill_filename, "w", zipfile.ZIP_DEFLATED) as zipf: # Walk through the skill directory for file_path in skill_path.rglob("*"): if file_path.is_file(): # Calculate the relative path within the zip arcname = file_path.relative_to(skill_path.parent) zipf.write(file_path, arcname) print(f" Added: {arcname}") ``` The validator only checks `SKILL.md` frontmatter and returns success without inspecting the package manifest: ```python def validate_skill(skill_path): """Basic validation of a skill""" skill_path = Path(skill_path) skill_md = skill_path / "SKILL.md" if not skill_md.exists(): return False, "SKILL.md not found" # Frontmatter validation omitted return True, "Skill is valid!" ``` ### Technical Analysis The packager recursively includes every path for which `Path.is_file()` returns true. It has no file allowlist, secret-file denylist, maximum file size, manifest review, or symbolic-link rejection. `Path.is_file()` follows symbolic links. Consequently, a file symlink located inside the Skill directory can refer to a readable file outside that directory. `zipfile.write()` then reads the target content and stores it under the symlink's relative archive name. Even without symlinks, the broad recursive inclusion can package unintended content such as: - `.env` files. - Authentication tokens. - Private keys. - Local configuration. - Evaluation artifacts containing sensitive prompts. - Version-control metadata. - Editor backups. - Build outputs and caches. The existing validation routine does not inspect any packaged file other than `SKILL.md`; therefore, successful validation does ...[truncated 1380 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Reject every symbolic link encountered during packaging. 2. Resolve each candidate path and verify it remains beneath the canonical Skill root using a safe containment check. 3. Package only explicitly supported top-level files and directories, such as `SKILL.md`, `_meta.json`, `scripts/`, `references/`, `assets/`, and approved evaluation files. 4. Exclude `.env`, private-key formats, credential files, VCS metadata, caches, editor backups, temporary files, and operating-system metadata. 5. Add per-file and total-archive size limits. 6. Scan candidate files for common secret patterns before archive creation. 7. Generate and display a complete archive manifest before writing or publishing the package. 8. Require explicit confirmation when unusual files, binary files, or hidden files are present. 9. Extend `quick_validate.py` to validate the complete package tree rather than only frontmatter. 10. Add automated tests covering external symlinks, internal symlinks, hidden secret files, oversized files, and nested unexpected directories. ]]>
Vulnerability Patterns
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (14)

Tp4

High
Category
MCP Tool Poisoning
Confidence
96% confidence
Finding
The code chunk implements a narrow packaging utility: it checks that a skill directory exists, requires SKILL.md, runs validate_skill, and zips the folder into a .skill file. This partially matches the declared 'package skill' capability, but does not support most of the much broader described functions such as creation, evaluation, benchmarking, pattern analysis, synthesis, or publishing. The primary behavior is therefore materially narrower than the declared purpose, so this is a description-behavior mismatch.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding
The declared description promises a broad skill authoring and lifecycle management capability, including creation, evaluation, improvement, benchmarking, synthesis, packaging, and publishing. The supplied code only performs basic validation of a single skill's SKILL.md frontmatter and related formatting rules. This is a materially narrower and different primary purpose than declared. There is no evidence of benchmarking, analysis, synthesis, packaging, publishing, or even general skill generation/editing behavior. Therefore the description does not accurately represent the actual code behavior.

Ae1

High
Category
analysis-evasion
Content
| **Synthesize** | Build skill from patterns | Scaffolded `SKILL.md` |
Confidence
100% confidence
Finding
Referenced artifact was not completely inspected

Self-Modification

High
Category
Rogue Agent
Content
_meta.json        # Auto-populated on publish
```

### Step 4 — Write SKILL.md

**Frontmatter rules:**
```yaml
Confidence
94% confidence
Finding
The skill explicitly instructs creation and modification of SKILL.md and related project artifacts, enabling self-modification or generation of executable agent behavior. In a skill-building context this is expected, but it is still high risk because a triggered run can persist new instructions, alter future agent behavior, or implant unsafe logic into other skills.

Self-Modification

High
Category
Rogue Agent
Content
Validates structure, outputs `<skill-name>.skill` zip.

### Step 6 — Iterate
Run evals (Mode 2) → identify failures → update SKILL.md → re-package → repeat.

---
Confidence
95% confidence
Finding
The iterative step to update SKILL.md and re-package enables repeated modification of persistent agent instructions. This increases risk because the skill can continuously rewrite behavior based on prior outputs or eval results, potentially entrenching unsafe changes or poisoning downstream published artifacts.

Self-Modification

High
Category
Rogue Agent
Content
# Print next steps
    print(f"\n[OK] Skill '{skill_name}' initialized successfully at {skill_dir}")
    print("\nNext steps:")
    print("1. Edit SKILL.md to complete the TODO items and update the description")
    if resources:
        if include_examples:
            print("2. Customize or delete the example files in scripts/, references/, and assets/")
Confidence
85% confidence
Finding
Skill modifies its own code, configuration, or behavior at runtime. Self-modification enables an agent to escalate privileges, disable safety constraints, or install persistent backdoors.

Lp3

Medium
Category
MCP Least Privilege
Confidence
91% confidence
Finding
The skill invokes shell commands, reads and writes files, and performs network publishing, but it declares no explicit tool or permission scope. That makes the skill over-privileged by default and increases the chance it will be triggered in contexts where destructive or sensitive operations were not intended.

Vague Triggers

Medium
Confidence
95% confidence
Finding
The trigger phrases are extremely broad, including common requests like 'create a skill', 'build a skill', and 'improve this skill', which raises the chance of accidental invocation. Because the skill can write files, run shell commands, inspect installed skills, and publish artifacts, unintended triggering could cause unreviewed modifications or data exposure.

Session Persistence

Medium
Category
Rogue Agent
Content
_meta.json        # Auto-populated on publish
```

### Step 4 — Write SKILL.md

**Frontmatter rules:**
```yaml
Confidence
84% confidence
Finding
Writing SKILL.md creates persistent state that affects future sessions and future skill invocations. In this context persistence is part of the feature, but without boundaries it can store unsafe instructions or silently change future behavior beyond the current interaction.

Session Persistence

Medium
Category
Rogue Agent
Content
Measure skill quality against defined expectations.

### Setup
Create `evals/evals.json`:
```json
[
  {
Confidence
82% confidence
Finding
Creating eval files and run histories introduces persistent artifacts that can influence later decisions, benchmarks, or improvement loops. While normal for an evaluation workflow, persistence becomes risky if untrusted prompts, assertions, or prior run outputs are later reused without validation.

Vague Triggers

Medium
Confidence
95% confidence
Finding
This manifest-like JSON assertion says descriptions should include trigger phrases like 'disk' or 'monitor'. In particular, 'monitor' is a broad everyday term, and the file provides no scope limits, negative examples, or disambiguation to prevent accidental invocation.

Vague Triggers

Medium
Confidence
96% confidence
Finding
The phrase 'you need speed' is highly vague and could spuriously match unrelated conversations, leading to accidental invocation of a high-performance browser automation skill. Although the phrase appears in a reference/patterns document rather than executable code, documenting such a trigger as a reusable pattern can normalize unsafe trigger design and cause insecure reuse in future skills.

Vague Triggers

Medium
Confidence
95% confidence
Finding
The listed trigger phrases for the skill creator include broad, natural-language requests such as 'create a skill', 'build a skill', and 'make a skill', which can match many benign user requests and cause unintended invocation. In a skill that can create, evaluate, package, and publish other skills, overbroad activation increases the chance of unsafe automation, unintended tool use, or propagation of insecure/generated skills.

Session Persistence

Medium
Category
Rogue Agent
Content
Options:
    --scan-dirs   Comma-separated list of skill directories to scan
                  Default: ~/.openclaw/workspace/skills/,~/.nvm/.../openclaw/skills/
    --output      Write report to this path (default: stdout)
    --query       Filter patterns relevant to a search term
"""
Confidence
60% confidence
Finding
Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.

Static analysis

No suspicious patterns detected.