T02 · Agent Memory Poisoning
Error
- Location
- SKILL.md:53
- Finding
- Untrusted User Corrections Can Be Promoted into Persistent Agent Instructions## Vulnerability Details **File Location**: `SKILL.md`, lines 53–245 **Vulnerability Type**: Persistent agent memory poisoning **Risk Level**: Critical ### Vulnerable Code Snippets At line 53, user-controlled corrections are designated as learning records: ```markdown | 2 | User correction ("incorrect"/"should be..."/"Actually...") | `LEARNINGS.md` | `correction` | ``` At lines 205–218, sufficiently important learnings are written into permanent agent instruction files: ```markdown When a learning is sufficiently important and general, refine it and write it into a permanent file: | Learning type | Promote to | Example | |---|---|---| | Workflow improvement | `AGENTS.md` | "Spawn each paper independently during batch processing" | | Tool usage technique | `TOOLS.md` | "Use a 3-second interval for the Semantic Scholar API rate limit" | | Behavioral pattern | `SOUL.md` | "Use unclassified/ when classification is uncertain" | Promotion steps: 1. Refine: condense the lengthy learning into a concise rule 2. Write: add it to the appropriate section of the target file 3. Update the original entry: Status → `promoted`, and fill in `Promoted-To` 4. Record a log: append a `promote` record to CHANGELOG.md ``` At lines 221–229, repeated patterns trigger automatic promotion: ```markdown When recording a new entry, first search for similar old entries: ```bash grep -r "keyword" .learnings/ ``` - If a similar entry is found → add reciprocal `See Also` links - **If the same pattern occurs ≥3 times** → trigger automatic promotion and write it into a permanent file - Repeated occurrence indicates that it is not an isolated event and should be solidified as a rule ``` At lines 238–249, persisted learnings can also be converted into new Skills: ```markdown When a learning meets any of the following conditions, it can be extracted as an independent skill: | Condition | Description | |---|---| | Has 2+ See Also links | Similar problems repeatedly occur | | Status is r ...[truncated 3237 chars]
- Remediation
- ## Remediation Suggestions 1. **Remove automatic promotion of user-derived content.** Never write corrections, external content, or task observations directly into `AGENTS.md`, `TOOLS.md`, `SOUL.md`, or another Skill without explicit human approval. 2. **Separate records from instructions.** Store learnings as inert data in a structured format that is not automatically loaded as agent policy. Do not interpret Markdown recorded from users as trusted instructions. 3. **Require approval for sensitive writes.** Present the proposed rule, its source, target file, and exact diff to an authorized reviewer. Require affirmative approval before modifying persistent agent-control files or creating Skills. 4. **Track provenance and trust.** Add mandatory fields identifying the source, author, session, trust level, and approval status. User corrections and externally sourced material must remain marked as untrusted. 5. **Enforce a policy allowlist.** Reject proposed rules that affect security controls, identity, permissions, secrets, data disclosure, external communications, command execution, or instruction precedence. 6. **Sanitize stored content.** Escape Markdown, XML, shell fragments, and prompt-like directives where appropriate. Apply strict size and character limits and prohibit embedded instructions from becoming executable context. 7. **Replace repetition with evidence-based validation.** Repetition must not establish trust. Require independent verification, tests, and reviewer approval regardless of how often a claim appears. 8. **Protect persistent files.** Restrict write permissions for `AGENTS.md`, `TOOLS.md`, `SOUL.md`, and `skills/` to a dedicated approval mechanism. The learning recorder should have append-only access solely to an inert audit store. 9. **Add rollback and audit controls.** Record immutable diffs, approving identities, timestamps, and provenance for every accepted promotion. Provide a reliable mechanism to revoke poisoned rules and identif ...[truncated 229 chars]