T02 · Agent Memory Poisoning
Warning
- Location
- SKILL.md:35
- Finding
- Untrusted User Corrections Are Written to Persistent Agent Memory<![CDATA[ ## Vulnerability Details **File Location**: `SKILL.md`, lines 35–49 and 72–74 **Vulnerability Type**: Persistent agent memory poisoning **Risk Level**: Medium ### Vulnerable Code ```markdown ## On User Corrections Trigger phrases (detect these): - "No, that's wrong / not right" - "Actually..." / "I prefer..." / "Remember that I..." - "Stop doing X" / "Why do you keep..." - "I told you before..." / "Always do X" When triggered: 1. Acknowledge the correction briefly 2. Append to `memory/corrections.md` under `## Corrections` with today's date 3. Call `memory_store` with: - category: `preference` (style/tone) or `decision` (behaviour/approach) - importance: 0.85 - text: `[CORRECTION] <what was wrong> → <correct behaviour>` + keywords 4. Recall to verify it stored correctly ``` ```markdown - **No secrets** — never log credentials, personal data, or sensitive info - **Corrections always log** — user corrections are always worth storing (importance ≥ 0.85) ``` ### Technical Analysis The Skill treats correction-like user input as trusted learning material and automatically writes it to two persistent locations: `memory/corrections.md` and the LanceDB-backed memory store. The stored record is assigned a relatively high importance of `0.85` and is immediately recalled for verification. No approval step, trust boundary, normalization process, provenance tracking, session scope, expiration policy, or validation against instruction-like content is specified. Consequently, an attacker can phrase a malicious behavioral directive as a correction, causing attacker-controlled rules to survive beyond the current interaction and potentially influence future sessions when memory is recalled. The unconditional requirement that corrections “always log” also conflicts with the separate prohibition on logging secrets and sensitive data. If a correction contains credentials, personal information, or other confidential content, the instructions do not define w ...[truncated 1722 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Require explicit user confirmation before persisting any correction beyond the current session. 2. Store a normalized factual summary rather than copying arbitrary user text into memory. 3. Reject correction content containing executable directives, attempts to alter safety constraints, tool-use rules, trust decisions, or instructions affecting unrelated future tasks. 4. Apply secret and personal-data detection before both filesystem and database writes. Redact sensitive values or refuse persistence when safe normalization is not possible. 5. Define an explicit precedence rule stating that the prohibition on storing secrets overrides the requirement to log corrections. 6. Record provenance with every entry, including user identity or trust domain, session identifier, timestamp, source message, and whether persistence was explicitly approved. 7. Scope preferences to the relevant user, workspace, and task category instead of applying them globally. 8. Treat recalled memories as untrusted contextual data rather than authoritative instructions. Recalled content must never override system policies, safety controls, or current user intent. 9. Add expiration, review, editing, and deletion controls for persistent records. 10. Avoid assigning high importance automatically. Importance should depend on confirmation, repetition, trusted provenance, and validation. 11. When showing learned patterns, redact sensitive data and restrict results to records authorized for the requesting user. 12. Add adversarial tests covering correction phrases that contain prompt injection, credentials, cross-session directives, and requests to weaken safety controls. ]]>