T02 · Agent Memory Poisoning
- Location
`. ``` `references/heartbeat-review.md:10-30` specifies the automated review and write process: ```markdown ### 2. Scan raw learnings Read `.learnings/LEARNINGS.md`, `.learnings/ERRORS.md`, and `.learnings/FEATURE_REQUESTS.md`. Count pending entries. ### 3. Evaluate recurrence For each pending entry: - Search all `.learnings/` files for similar entries (by keyword, domain, or pattern). - If a match exists: increment `Recurrence-Count` on the older entry, add `See Also` link. - If `Recurrence-Count >= 3 ...[truncated 4787 chars]:40- Finding
Untrusted Conversation Content Can Be Promoted into Persistent Agent Memory
- Content
View full analysis
Vulnerability Details
File Location:
SKILL.md:40-48, 63-73, 91-102, 116-143;references/heartbeat-review.md:10-30;references/promotion-rules.md:13-33;references/workspace-layout.md:37-48
Vulnerability Type: T02: Agent Memory Poisoning
Risk Level: HighVulnerable Code and Instructions
SKILL.md:40-48accepts conversation-derived preferences as learning entries:markdown | Signal | Target file | Category | |--------|-------------|----------| | User corrects you | `LEARNINGS.md` | `correction` | | User says "always/never do X" | `LEARNINGS.md` | `preference` | | You discover something non-obvious | `LEARNINGS.md` | `insight` | | Your knowledge was outdated | `LEARNINGS.md` | `knowledge_gap` | | Found a better approach | `LEARNINGS.md` | `best_practice` | | Command returns non-zero | `ERRORS.md` | — | | Tool/API fails unexpectedly | `ERRORS.md` | — | | User wants missing capability | `FEATURE_REQUESTS.md` | — |SKILL.md:63-73permits promotion into always-loaded and permanent behavior files:markdown ### 3. Promote Move validated patterns up the memory tiers: | Condition | Promote to | Example | |-----------|-----------|---------| | Pattern used 2x, context-specific | `.self-improving/domains/*.md` or `projects/*.md` | "This repo uses pnpm" | | Pattern used 3x in 7-30 days, cross-task | `.self-improving/HOT.md` | "User prefers concise answers" | | Stable, long-term applicable | `SOUL.md` / `AGENTS.md` / `TOOLS.md` / `MEMORY.md` | "Never force push" | See [references/promotion-rules.md](references/promotion-rules.md) for the full promotion/demotion rules.SKILL.md:116-143definesHOT.mdas automatically loaded and identifies permanent promotion targets:markdown ## Memory Tiers | Tier | Location | Size limit | Behavior | |------|----------|-----------|----------| | RAW | `.learnings/` | Unlimited | Intake only, not auto-loaded | | HOT | `.self-improving/HOT.md` | ≤80 lines | Always loaded at session sta ...[truncated 6070 chars]- Remediation
View remediation
Remediation Suggestions
-
Require explicit approval for promotion
- Never automatically promote conversation-derived content into
HOT.md,SOUL.md,AGENTS.md,TOOLS.md, orMEMORY.md. - Display the exact proposed text, source entries, destination file, and diff to a trusted user.
- Require an affirmative confirmation for each promotion.
- Never automatically promote conversation-derived content into
-
Track and enforce provenance
- Record the actor, conversation, timestamp, trust level, and original text for every learning.
- Allow permanent promotion only when all supporting observations originate from an authenticated, authorized user.
- Do not count repetition by the same untrusted source as independent corroboration.
-
Separate data from instructions
- Store conversation-derived content as quoted, non-executable data.
- Generate any actionable rule through a constrained transformation rather than copying user text verbatim.
- Prevent raw text from being inserted into files interpreted as agent instructions.
-
Validate proposed rules
- Reject or quarantine rules that request bypassing safety constraints, changing permissions, exposing secrets, suppressing confirmation, executing commands, downloading content, or altering security controls.
- Apply allowlisted schemas for benign preference categories.
- Send security-sensitive candidates to manual review regardless of recurrence.
-
Limit writable destinations
- Prohibit automated writes to
SOUL.md,AGENTS.md, andTOOLS.md. - Use a dedicated, lower-trust memory file whose entries cannot override system, developer, workspace, or security instructions.
- Enforce precedence so learned memory is always subordinate to trusted policies.
- Prohibit automated writes to
-
Add integrity and rollback controls
- Keep an append-only audit log of promotions.
- Version memory files and support immediate rollback.
- Periodically detect unexpected changes to permanent agent-control files.
-
**Harden heartbeat processi ...[truncated 258 chars]
-
