T02 · Agent Memory Poisoning
Warning
- Location
- SKILL.md:31
- Finding
- Automatic Persistent Storage of Unverified Behavioral Inferences<![CDATA[ ## Vulnerability Details **File Location**: `SKILL.md`, lines 31–54 and 117–123 **Vulnerability Type**: Persistent memory poisoning through unverified behavioral inference **Risk Level**: Medium ### Complete Code Snippet ```markdown ## Capture Triggers Log memory when any of these happen: - user says “remember this” - a preference appears repeatedly - a boundary is stated (“don’t do X”, “keep Y private”) - a recurring blocker/pattern emerges - project priorities shift meaningfully ## Memory Tiers - **Daily notes**: `memory/YYYY-MM-DD.md` - timestamped raw events, short and factual - **Long-term**: `MEMORY.md` - curated durable profile and preferences ## Write Workflow 1. Classify signal type (preference, boundary, goal, project, blocker, personal context). 2. Append concise timestamped entry to daily memory. 3. Form 1–2 human-like assumptions (hypotheses) from behavior patterns. 4. Tag each assumption with confidence (high/medium/low). 5. Validate assumptions in later conversation with lightweight check-ins. 6. Promote validated, durable items to long-term memory. ``` Related inference instructions at lines 117–123: ```markdown ## Assumption Loop (Human-Like Understanding) For deeper understanding, run this loop continuously: 1. Observe behavior pattern (not just words). 2. Infer a tentative assumption about the user. 3. Store assumption as hypothesis (never as fact initially). 4. Test it with a small conversational probe. 5. Update confidence or discard if contradicted. ``` ### Technical Analysis The skill directs the agent to persist preferences, recurring patterns, blockers, and inferred behavioral characteristics without requiring explicit consent for each write. Explicit consent is only preferred for sensitive information, while ordinary behavioral inferences may be written automatically. Although inferred entries are labeled as hypotheses, they are still placed in persistent daily memory and can later be promoted to `MEMORY.md` ...[truncated 2174 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Require explicit user opt-in before every persistent memory write, including preferences, boundaries, behavioral patterns, and hypotheses. 2. Keep inferred hypotheses in ephemeral session state by default. Persist them only after the user confirms both their accuracy and intended retention. 3. Introduce a strict memory schema that permits descriptive user context but rejects: - Instructions to ignore system or developer policies. - Claims that grant authorization or elevated privileges. - Tool-use or command-execution directives. - Secrets, credentials, tokens, and unnecessary sensitive data. - Content copied from untrusted documents, websites, or third parties. 4. Record provenance for every entry, including whether it was explicitly stated, inferred, supplied by a third party, or extracted from external content. 5. Require separate confirmation before promoting an entry from daily memory to `MEMORY.md`. 6. Present stored content to the user for review and provide accessible deletion, correction, and export controls. 7. Apply retention limits to all unconfirmed entries. Delete rather than merely downgrade hypotheses that expire without confirmation. 8. Treat persistent memory as untrusted data when retrieved. Never interpret a stored memory entry as a higher-priority instruction. 9. Add automated tests demonstrating that preference-like prompt injections and operational directives cannot enter durable memory. ]]>
