Back to skill

Security audit

Learning Loop

Security checks for vulnerabilities and agentic risk

Overview

This skill is coherent for persistent agent memory, but it needs review because it can permanently change agent behavior from imported rules and includes an unsafe script argument bug.

Install only if you want persistent agent memory that can influence future sessions. Keep the memory directory local, avoid storing secrets or sensitive personal/business data, disable broad per-message capture unless you need it, manually review every imported rule even after a dry run, and avoid passing untrusted workspace or filter values to inject-rules.sh until that script is fixed.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (2)

T01 · Skill Instruction Hijacking

Error
Location
import-rules.sh:175
Finding
Unauthenticated Rule Imports Enable Persistent Agent Instruction Poisoning<![CDATA[ ## Vulnerability Details **File Location**: `import-rules.sh:175-201, 203-249, 278-378`; downstream enforcement in `inject-rules.sh:30-46` and `SKILL.md:76-86, 110-120` **Vulnerability Type**: Unauthenticated persistent behavioral-rule import **Risk Level**: Critical ### Vulnerable Code ```python def calculate_trust_score(import_metadata, imported_rules): """Calculate trust score for the import based on various factors.""" score = 0.5 # Base score # Factor 1: Number of rules (more rules = more established) rule_count = len(imported_rules) if rule_count >= 20: score += 0.1 elif rule_count >= 10: score += 0.05 # Factor 2: Average confidence of imported rules avg_confidence = sum(r.get("confidence_score", 0.9) for r in imported_rules) / len(imported_rules) if imported_rules else 0 score += avg_confidence * 0.2 # Factor 3: Category diversity categories = set(r.get("category") for r in imported_rules) if len(categories) >= 5: score += 0.1 elif len(categories) >= 3: score += 0.05 # Factor 4: Has manifest hash (integrity verification) if import_metadata.get("manifest_hash"): score += 0.05 return min(1.0, score) ``` The resulting score controls automatic import: ```python trust_score = calculate_trust_score(metadata, imported_rules) print(f"Trust score: {trust_score:.2f} (threshold: {trust_threshold})") if trust_score < trust_threshold: print(f"\n⚠️ WARNING: Trust score {trust_score:.2f} is below threshold {trust_threshold}") print("Import will proceed in review mode (all rules need manual approval)") review_mode = True else: review_mode = False ``` Attacker-provided rule text is copied into the persistent rule store: ```python for imported_rule in imported_rules: original_id = imported_rule.pop("_original_id", None) rule_hash = imported_rule.pop("_hash", None) conflicts = detect_conflict(imported_rule, existing_rul ...[truncated 5394 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Eliminate automatic import of behavioral instructions. Require explicit, per-rule human approval before any imported rule is written to the active rule store. 2. Default every unknown or unsigned source to untrusted, regardless of rule count, category diversity, or claimed confidence. 3. Establish an allowlist of trusted publisher identities and verify exports with authenticated digital signatures. A bare SHA-256 digest only provides integrity when the expected digest is obtained through a trusted channel. 4. Recalculate every rule hash from a canonical serialization and reject any mismatch. 5. Recalculate and verify the complete manifest hash instead of awarding trust merely because the field exists. 6. Do not default missing confidence to `0.9`. Reject missing or malformed fields under a strict schema. 7. Validate `type`, `category`, `rule`, `reason`, confidence bounds, maximum lengths, and object types before processing. 8. Place accepted imports in a quarantined review file rather than the active `rules.json`. 9. Distinguish trusted local rules from imported reference material. Never label unreviewed external text as instructions that the agent must follow. 10. Add semantic policy checks for rules requesting secret disclosure, safety bypasses, tool execution, instruction precedence changes, or unauthorized communications. 11. Preserve immutable provenance and provide a command to revoke or roll back every rule from a particular import. 12. Add adversarial tests covering fake hashes, self-asserted confidence, plain-array imports, contradictory wording, prompt-injection text, and malformed rule objects. ]]>

T09 · Insecure Skill Coding Practices

Error
Location
inject-rules.sh:16
Finding
Python Code Injection Through Shell-Interpolated Workspace and Filter Arguments<![CDATA[ ## Vulnerability Details **File Location**: `inject-rules.sh:16-24` **Vulnerability Type**: Code injection through unsafe construction of Python source **Risk Level**: High ### Vulnerable Code ```bash python3 -c " import json, sys with open('$RULES_FILE') as f: data = json.load(f) rules = data.get('rules', []) filt = '$FILTER' if filt != 'all': cats = set(filt.split(',')) rules = [r for r in rules if r['category'] in cats] ``` The affected variables originate from command-line arguments: ```bash WORKSPACE="${1:-/Users/gregborden/.openclaw/workspace}" FILTER="${2:-all}" # comma-separated categories, or "all" RULES_FILE="$WORKSPACE/memory/learning/rules.json" ``` ### Technical Analysis `RULES_FILE` and `FILTER` are inserted directly into a double-quoted `python3 -c` program and then placed inside single-quoted Python string literals. Shell quoting does not make these values safe Python syntax. An attacker who can control either argument can supply a single quote followed by valid Python syntax, close the intended string, and inject additional statements or expressions. The resulting code runs under `python3` with the same operating-system identity and privileges as the Skill. This is not limited to altering the category filter. Python's standard library permits process execution, file access, environment inspection, and network access where allowed by the host. A crafted workspace value must also resolve through the script's preliminary file-existence check, whereas the filter argument reaches the vulnerable Python assignment directly after that check. ### Attack Path 1. The attacker gains influence over the category-filter argument passed to `inject-rules.sh`. This can occur when an agent builds the command from untrusted task content or configuration. 2. The attacker supplies a value containing a quote and injected Python syntax. 3. The shell expands `$FILTER` inside the source passed to `python3 -c`. 4. The injected quote ter ...[truncated 1009 chars]
Remediation
<![CDATA[ ## Remediation Suggestions Pass data as arguments instead of embedding it into Python source. Use a quoted heredoc so the shell cannot interpolate untrusted values: ```bash python3 - "$RULES_FILE" "$FILTER" <<'PY' import json import sys rules_file, filt = sys.argv[1], sys.argv[2] with open(rules_file, encoding="utf-8") as f: data = json.load(f) rules = data.get("rules", []) if filt != "all": categories = set(filt.split(",")) rules = [r for r in rules if r.get("category") in categories] groups = {"MUST": [], "NEVER": [], "CHECK": [], "PREFER": []} for rule in rules: rule_type = rule.get("type", "CHECK") if rule_type in groups: groups[rule_type].append(rule) PY ``` Additional hardening should include: 1. Validate filters against a strict category allowlist and reject control characters or unexpected syntax. 2. Canonicalize the workspace path and ensure it remains under an approved workspace root. 3. Avoid the hardcoded user-specific default workspace path; require an explicit workspace or default to the current directory. 4. Apply maximum lengths to rule text and filter arguments. 5. Add automated tests using quotes, newlines, backslashes, shell metacharacters, and Python syntax in both arguments. 6. Run the injector under the least-privileged account available and without unnecessary network or credential access. ]]>
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • System Prompt LeakageDirect Leakage, Indirect Extraction, Tool-Based Exfiltration
  • Tool MisuseTool Parameter Abuse, Chaining Abuse, Unsafe Defaults
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
Findings (38)

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding
The declared description presents a comprehensive self-improvement memory system with multiple analytical and operational capabilities. The supplied code chunk does not implement those behaviors; it only performs housekeeping on an events log by moving entries older than a retention threshold into monthly archive files. While archiving old learning events could be a supporting part of persistent memory management, it does not substantiate the broad declared functionality such as confidence decay, anomaly detection, cross-agent sharing, or review/promote workflows. Therefore the code's actual primary purpose is materially narrower and different from the declared description.

Tp4

High
Category
MCP Tool Poisoning
Confidence
93% confidence
Finding
The supplied code implements only one subset of the declared system: confidence decay plus review flagging for rules and lessons stored in local JSON files. It does not perform cross-agent sharing, anomaly detection, risky-action prechecks, promotion of patterns to enforced rules, or any persistence/compaction logic beyond editing existing files in a workspace. It also does not capture new lessons from debugging sessions or user feedback; it merely updates and reports on existing records. Because the description presents a substantially broader skill than the actual code supports, this is a material description-behavior mismatch.

Tp4

High
Category
MCP Tool Poisoning
Confidence
91% confidence
Finding
The code partially matches the description: it clearly supports weekly review of metrics, rule-promotion suggestions, anomaly detection, and self-improvement reporting over persistent local files. However, the declared description presents a broader self-improvement system with confidence decay, cross-agent sharing, pre-risk rule checking, and persistent memory setup. This specific code chunk does not implement those capabilities; it is narrowly a weekly analysis/report generator over existing memory artifacts. Because the declared purpose materially overstates the behavior of the supplied code chunk, the description does not accurately represent what this code actually does.

Tp4

High
Category
MCP Tool Poisoning
Confidence
93% confidence
Finding
The description claims a broad structured self-improvement system with several substantive capabilities: confidence decay, cross-agent sharing, anomaly detection, risky-action rule checks, weekly review metrics, and persistent memory support. The supplied code does only a narrow subset: it scans a daily log for certain keywords, prints relevant excerpts, initializes an events file if absent, and tells the operator what to do next. It does relate loosely to capturing lessons learned and persistent memory files, so it is not wholly unrelated. However, the primary behavior is a daily extraction helper, and the major advertised capabilities are absent from the code chunk. That makes the description materially overstated relative to actual behavior.

Tp4

High
Category
MCP Tool Poisoning
Confidence
95% confidence
Finding
The code chunk implements only one narrow component: inline detection and logging of feedback-related signals from a message. It validates a workspace, loads or falls back to phrase patterns, scores matches, prints a JSON result, and may append an event to memory/learning/events.jsonl. That is plausibly related to the declared use case of capturing feedback or lessons learned, but it does not substantiate most of the description's major claims. There is no evidence of confidence decay, cross-agent sharing, anomaly detection, persistent memory beyond simple local event logging, risky-action checks, weekly review metrics, or promotion of patterns into enforced rules. The declared description therefore materially overstates the capability and purpose of this specific code chunk.

Tp4

High
Category
MCP Tool Poisoning
Confidence
92% confidence
Finding
The code only implements a guard/checkpoint component of a larger learning system: it reads rules.json and lessons.json, scores relevant rules for a proposed action, logs guard events to guard-log.jsonl, and in --save mode credits related lessons by incrementing times_saved. This partially aligns with the declared trigger 'Before risky actions to check relevant rules' and with tracking saves/metrics. However, the declared description emphasizes a broader self-improvement system with confidence decay, cross-agent sharing, anomaly detection, lesson capture after debugging/feedback, weekly review, and persistent memory behavior. None of those major capabilities appear in this code chunk. Because the actual code's primary behavior is much narrower than the declared purpose, the description does not accurately represent this supplied code chunk.

Tp4

High
Category
MCP Tool Poisoning
Confidence
95% confidence
Finding
The declared description presents a broad agent self-improvement and persistent memory framework with multiple learning and monitoring features. The supplied code chunk does not implement those behaviors. Instead, it is a simple utility that loads existing rules from a JSON file, optionally filters them, groups them by type, and prints a compact text block for downstream agents to read. While this loosely relates to 'checking relevant rules' and persistent memory, the actual code is much narrower and materially different in primary purpose. Therefore, the description does not accurately represent what this code chunk actually does.

Tp4

High
Category
MCP Tool Poisoning
Confidence
83% confidence
Finding
The code is related to one part of the declared system—weekly/daily promotion of proven lessons into rules—so it is not wholly unrelated. However, the declared description presents a broader self-improvement platform with multiple major features (confidence decay, cross-agent sharing, anomaly detection, risky-action rule checking, persistent memory management). This code chunk only implements the rule-promotion pipeline over local JSON files. Because the actual behavior is materially narrower than the declared purpose and omits several prominently claimed capabilities, the description does not accurately represent what this supplied code chunk actually does.

Tp4

High
Category
MCP Tool Poisoning
Confidence
91% confidence
Finding
The declared description presents a comprehensive self-improvement framework with multiple capabilities: capturing lessons, incorporating feedback, persistent memory across compactions, confidence decay, cross-agent sharing, anomaly detection, and periodic review/promotion of patterns into rules. The supplied code does not implement that system. It only loads memory/learning/rules.json and retrieves relevant rules for a provided action description using heuristic scoring. This does align with one narrow declared use case—checking relevant rules before risky actions—but it does not support the broader declared purpose and omits the headline features entirely. Therefore the description materially overstates what this code chunk actually does.

Tp4

High
Category
MCP Tool Poisoning
Confidence
90% confidence
Finding
The description claims a full self-improvement system with several substantive capabilities: confidence decay, cross-agent sharing, anomaly detection, pre-action checks, weekly promotion workflows, and persistent memory setup/use. The supplied code does something narrower and different: it audits whether a separate learning-loop system appears healthy by inspecting files and metrics in memory/learning, checking freshness/completeness/count thresholds, reading AGENTS.md and HEARTBEAT.md for references, inspecting guard logs, and producing a score plus recommendations. While the weekly review aspect loosely overlaps, most headline capabilities in the declaration are absent from this code. The code neither implements confidence decay nor cross-agent sharing, does not perform actual rule enforcement before risky actions, and does not capture or persist memories itself. Therefore the declared description materially overstates and mischaracterizes the code’s actual purpose and behavior.

Tp4

High
Category
MCP Tool Poisoning
Confidence
95% confidence
Finding
The code chunk is narrowly focused on automatic rule-violation tracking within a local learning-memory directory. It reads events and rules files, identifies mistake/bug-fix events, scores matches against existing rules, increments per-rule violation counts, and logs parse errors. This is at best a small supporting component of a larger self-improvement system, but it does not implement most of the headline capabilities in the description: there is no confidence decay, cross-agent sharing, anomaly detection, weekly review/promotion, or persistent-memory management beyond reading/writing local files. The declared purpose therefore materially overstates and mischaracterizes the actual behavior. The daily-cron nature of the script also differs from the listed use cases/triggers.

Tp4

High
Category
MCP Tool Poisoning
Confidence
95% confidence
Finding
The supplied code chunk is much narrower than the declared description. It only updates metrics from existing local files and writes metrics.json, plus parse error logging and locking. It does fit one declared use case—weekly review of metrics—but it does not implement the broader advertised capabilities such as confidence decay, cross-agent sharing, anomaly detection, capturing lessons from debugging/feedback, checking rules before risky actions, or managing persistent memory across session compactions. Thus the description materially overstates the behavior of this code chunk.

Tp4

High
Category
MCP Tool Poisoning
Confidence
94% confidence
Finding
The code implements only a narrow local logging mechanism: it writes structured notes into SESSION-STATE.md and updates a last-updated timestamp. That partially aligns with the description's persistent memory and capturing lessons/corrections aspects, but the declared description claims a much broader self-improvement system with confidence decay, cross-agent sharing, anomaly detection, periodic review, and rule promotion. None of those capabilities are present in this code chunk. The primary behavior is local session-state persistence, not a full structured self-improvement system as described.

Tool Parameter Abuse

High
Category
Tool Misuse
Content
**Solution:**
1. Check for stuck processes: `lsof memory/learning/.lockfile`
2. Kill stuck process if safe: `kill <pid>`
3. Remove stale lock file: `rm memory/learning/.lockfile`
4. Re-run the script

### Confidence decay not working
Confidence
85% confidence
Finding
The documentation advises manually removing a lock file with `rm memory/learning/.lockfile` after only a brief safety check. While not inherently malicious, manual lockfile deletion can lead to race conditions, concurrent writes, corruption, or accidental misuse if copied blindly in the wrong context or path.

Direct Prompt Extraction

High
Category
System Prompt Leakage
Content
echo "  --help, -h          Show this help message"
            echo ""
            echo "Examples:"
            echo "  bash export-rules.sh /path/to/workspace --output rules-export.json"
            echo "  bash export-rules.sh --category shell > shell-rules.json"
            exit 0
            ;;
Confidence
85% confidence
Finding
Skill contains instructions that could directly expose system prompts, internal rules, or hidden instructions to users or external parties.

Direct Prompt Extraction

High
Category
System Prompt Leakage
Content
echo "  --help, -h          Show this help message"
            echo ""
            echo "Examples:"
            echo "  bash export-rules.sh /path/to/workspace --output rules-export.json"
            echo "  bash export-rules.sh --category shell > shell-rules.json"
            exit 0
            ;;
Confidence
85% confidence
Finding
Skill contains instructions that could directly expose system prompts, internal rules, or hidden instructions to users or external parties.

Direct Prompt Extraction

High
Category
System Prompt Leakage
Content
echo "  --help, -h          Show this help message"
            echo ""
            echo "Examples:"
            echo "  bash export-rules.sh /path/to/workspace --output rules-export.json"
            echo "  bash export-rules.sh --category shell > shell-rules.json"
            exit 0
            ;;
Confidence
85% confidence
Finding
Skill contains instructions that could directly expose system prompts, internal rules, or hidden instructions to users or external parties.

Vague Triggers

High
Confidence
95% confidence
Finding
The positive 'subtle' triggers include very common conversational phrases such as 'yeah', 'sure', 'do it', and 'go ahead' that can appear in normal dialogue without indicating meaningful satisfaction. In a self-improvement or memory-writing skill, this can cause accidental promotion of ordinary exchanges into persisted feedback signals, poisoning learning data and reinforcing the wrong behaviors over time.

Vague Triggers

High
Confidence
97% confidence
Finding
The correction and preference lists contain extremely broad directives such as 'always', 'never', 'from now on', 'i want', and 'go with', which can occur in ordinary planning or task instructions rather than durable policy feedback. Because the skill is designed for self-improvement and persistent memory, these patterns risk converting transient user phrasing into lasting rules or preferences, enabling instruction poisoning and policy drift.

Ssd 3

High
Confidence
98% confidence
Finding
The file mandates logging broad categories of user-provided information before responding, including 'specific values' and examples like expiration details, which could easily include secrets, personal data, or operationally sensitive information. The surrounding skill context makes this more dangerous because it is explicitly designed for persistent self-improvement memory that survives compaction and supports cross-agent sharing, expanding both retention and exposure scope.

Natural-Language Policy Violations

Medium
Confidence
89% confidence
Finding
The changelog documents a design where the agent must scan every message for WAL triggers, which implies persistent monitoring of all user inputs without explicit opt-in or clear minimization boundaries. In a learning/persistent-memory skill, this increases the chance of over-collection of sensitive data, unintended retention of private content, and behavioral drift toward always-on surveillance of conversations.

Lp3

Medium
Category
MCP Least Privilege
Confidence
91% confidence
Finding
The skill declares broad operational capabilities via required binaries and documented shell workflows, but does not declare an explicit tool/permission scope. That makes it harder to enforce least privilege and increases the chance an agent can read, write, execute shell commands, or perform sharing behaviors beyond what a user expects.

Ssd 3

Medium
Confidence
95% confidence
Finding
The skill promotes persistent retention of user feedback and session-derived knowledge across sessions and even across agents. Long-lived memory of raw interactions increases the chance that secrets, personal data, or sensitive business context are stored indefinitely and later exposed or reused out of context.

Ssd 3

Medium
Confidence
96% confidence
Finding
The append-only guidance for raw event logs encourages indefinite retention of debugging sessions, mistakes, successes, and feedback. Append-only logs are useful operationally, but without retention, filtering, or redaction controls they become a durable sink for sensitive data and increase breach impact.

Missing User Warnings

Medium
Confidence
94% confidence
Finding
The skill explicitly supports cross-agent export/import of rules and knowledge, but does not warn that exported rules may contain sensitive operational history, user-derived guidance, or environment-specific secrets. Sharing persistent memory artifacts across agents can leak confidential information or propagate unsafe rules into other environments.