T02 · Agent Memory Poisoning
- Location
HEARTBEAT.md:15- Finding
Untrusted User Corrections Can Be Promoted into Persistent Agent Memory
- Content
View full analysis
Vulnerability Details
File Location:
HEARTBEAT.md, lines 15-25
Vulnerability Type:T02: Agent Memory Poisoning
Risk Level: MediumVulnerable instructions:
markdown ## Weekly Tasks (Sunday) - [ ] Review `self-improving/corrections.md` and promote patterns repeated three or more times to `memory.md` (HOT layer) - [ ] Check pending improvements in `self-improving/projects/realestate-advisor.md` and update their progress - [ ] Remove archives older than 180 days from `self-improving/archive/` --- ## Event-Driven Triggers - **User correction** → Immediately record it in `self-improving/corrections.md`Technical Analysis
The workflow instructs the agent to persist user-supplied corrections immediately and later promote patterns that occur at least three times into long-term memory. It does not require owner approval, provenance verification, semantic validation, or separation of factual data from behavioral instructions.
Consequently, attacker-controlled text presented as a correction could cross the trust boundary from an untrusted conversation into persistent state. Repetition is the only stated promotion criterion and is not a meaningful authenticity check. If long-term memory is subsequently loaded as authoritative agent context, a promoted instruction-like entry could influence future sessions.
Exploitation depends on the heartbeat and memory-management instructions being active and on later sessions consuming the resulting memory. The audited package does not contain executable code that independently performs these writes.
Attack Path
- An attacker interacts with the agent and submits a crafted instruction disguised as a correction.
- The event-driven rule causes that content to be recorded in
self-improving/corrections.md. - The attacker repeats the correction at least three times, satisfying the stated promotion threshold.
- During the weekly heartbeat review, t ...[truncated 783 chars]
- Remediation
View remediation
Remediation Suggestions
- Require explicit, authenticated owner approval before any user-originated correction is written to persistent storage or promoted into long-term memory.
- Store corrections as inert, quoted records with metadata such as source, timestamp, conversation identifier, and approval status.
- Apply a strict schema that permits only narrowly defined factual corrections or user preferences; reject commands, tool instructions, policy changes, and role-modifying text.
- Replace repetition-based promotion with a controlled review process. Repeated attacker input must not be treated as evidence of trustworthiness.
- Keep untrusted correction records separate from authoritative instruction and memory files.
- Escape or delimit stored content so it cannot be interpreted as agent instructions when memory is loaded.
- Maintain an append-only audit trail for memory additions, approvals, modifications, and removals.
- Provide rollback and quarantine mechanisms for suspicious entries.
- Run future sessions with least-privilege tool access so memory poisoning cannot automatically translate into sensitive external actions.