T02 · Agent Memory Poisoning
Error
- Location
- SKILL.md:12
- Finding
- Unvalidated Corrections Can Poison Persistent Agent Memory<![CDATA[ ## Vulnerability Details **File Location**: `SKILL.md:12-16`, `SKILL.md:47-77`, and `SKILL.md:87-100` **Vulnerability Type**: Persistent memory poisoning through untrusted rule creation **Risk Level**: High ### Vulnerable Code Snippets ```markdown When an agent makes an error or gets corrected, it must: 1. Extract a **rule** (not a story) 2. Write it to `lessons.md` in its workspace 3. Scan relevant rules **before** future decisions in that domain 4. Optionally share anonymized rules to the community repo ``` ```markdown ## When to Record Record a rule when: 1. **User corrects you** — explicit feedback 2. **User overrides your output** — they redo your work 3. **Same error twice** — second occurrence MUST become a rule 4. **Near miss** — you catch yourself about to repeat a mistake Do NOT record: one-off technical glitches, user preference changes (those go in MEMORY.md). ## How to Record 1. Stop. Don't apologize at length. 2. Identify the category. 3. Write the rule in imperative form. 4. Append to lessons.md (never overwrite). 5. Confirm briefly: "Added to lessons: [title]" ``` ```markdown 3. Add to your startup instructions: ```markdown - On startup: skim top-100.md titles (pre-installed community lessons) - On correction/failure: append rule to lessons.md - Before decisions: scan lessons.md + top-100.md for [CATEGORY] rules ``` ``` ### Technical Analysis The skill treats user corrections, output overrides, and perceived mistakes as sources of persistent executable guidance. It directs the agent to append extracted rules to `lessons.md` and consult those rules before future decisions and on subsequent startups. The process does not require: - Approval before persistence. - Verification that a correction is accurate or trustworthy. - Provenance identifying the user or session that supplied the rule. - Restrictions preventing rules from weakening security or tool-use policies. - Project, user, or trust-boundary isolation. - Conflict detect ...[truncated 2081 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Treat every proposed lesson as untrusted data rather than authoritative instruction. 2. Require explicit, informed user approval before writing a new persistent rule. 3. Store provenance with each lesson, including source user, session, timestamp, affected project, and approval status. 4. Reject lessons that attempt to modify security controls, permission boundaries, confirmation requirements, credential handling, or instruction precedence. 5. Validate proposed lessons against a strict schema and an allowlist of acceptable behavioral domains. 6. Isolate lesson files by workspace, project, and user to prevent cross-context contamination. 7. Load lessons as advisory context only; explicitly state that they cannot override system, developer, safety, access-control, or tool-use policies. 8. Implement conflict detection and present conflicting rules for human review. 9. Add versioning, expiration, rollback, and audit logging for all lesson changes. 10. Provide a review queue so corrections are not automatically persisted during an adversarial interaction. ]]>