T02 · Agent Memory Poisoning
Warning
- Location
- operations.md:23
- Finding
- Persistent behavioral rules may be stored without explicit cross-session consent<![CDATA[ ## Vulnerability Details **File Location**: `SKILL.md:47-70`, `operations.md:23-35`, `learning.md:3-12`, `boundaries.md:52-59` **Vulnerability Type**: Persistent memory poisoning through automatic, insufficiently validated writes **Risk Level**: Medium ### Vulnerable Code Snippets `SKILL.md:47-70`: ```markdown ## Detection Triggers Log automatically when you notice these patterns: **Corrections** → add to `corrections.md`, evaluate for `memory.md`: - "No, that's not right..." - "Actually, it should be..." - "You're wrong about..." - "I prefer X, not Y" - "Remember that I always..." - "I told you before..." - "Stop doing X" - "Why do you keep..." **Preference signals** → add to `memory.md` if explicit: - "I like when you..." - "Always do X for me" - "Never do Y" - "My style is..." - "For [project], use..." **Pattern candidates** → track, promote after 3x: - Same instruction repeated 3+ times - Workflow that works well repeatedly - User praises specific approach ``` `operations.md:23-35`: ```markdown ### On Correction Received ``` 1. Parse correction type (preference, pattern, override) 2. Check if duplicate (exists in any tier) 3. If new: - Add to corrections.md with timestamp - Increment correction counter 4. If duplicate: - Bump counter, update timestamp - If counter >= 3: ask to confirm as rule 5. Determine namespace (global, domain, project) 6. Write to appropriate file 7. Update index.md line counts ``` ``` `learning.md:3-12`: ```markdown ## What Triggers Learning | Trigger | Confidence | Action | |---------|------------|--------| | "No, do X instead" | High | Log correction immediately | | "I told you before..." | High | Flag as repeated, bump priority | | "Always/Never do X" | Confirmed | Promote to preference | | User edits your output | Medium | Log as tentative pattern | | Same correction 3x | Confirmed | Ask to make permanent | | "For this project..." | Scoped | Write to project namespace | ``` `boundaries.md:52-59`: ...[truncated 3693 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Require explicit opt-in before the first write to any cross-session file. 2. Keep unconfirmed corrections and inferred observations session-local rather than writing them to `~/self-improving/`. 3. Present the exact proposed memory entry, its namespace, retention period, and source to the user before persistence. 4. Remove `User edits your output` as an automatic persistence trigger, or require explicit confirmation that the edit represents a reusable preference. 5. Add a mandatory memory-validation layer that rejects content involving: - Overrides of system, developer, safety, or authorization rules. - Requests to conceal actions or stored state. - Credentials, secrets, financial data, medical data, or access patterns. - Instructions to invoke unrelated tools, execute commands, or access resources. - Personal information about third parties. 6. Treat stored memory as untrusted data. It must never outrank system, developer, current-user, authorization, or safety constraints. 7. Require confirmation before promotion to HOT memory, even when a statement contains words such as “always” or “never.” 8. Remove the person-specific examples from `learning.md` and `scaling.md`, replacing them with non-personal project or document-format examples. 9. Add per-entry ownership, source, consent status, creation time, scope, and expiration metadata. 10. Add tests confirming that no persistent file is modified before opt-in and that unsafe preference text cannot be promoted. ]]>