T02 · Agent Memory Poisoning
Error
- Location
- SKILL.md:43
- Finding
- Persistent Agent Behavior Modification Through Untrusted Interaction Data<![CDATA[ ## Vulnerability Details **File Location**: `SKILL.md`, lines 43–54, with the behavior-update trigger at line 88 **Vulnerability Type**: T02: Agent Memory Poisoning **Risk Level**: High ### Vulnerable Code Snippet ```markdown File location: `memory/usage-logs/<skill-name>.md` ## Self-Iteration Triggers Evaluate these conditions during each **periodic review** (default daily, configurable): | Condition | Action | |-----------|--------| | 3+ consecutive successful invocations | Mark skill as "stable" — reduce context allocation | | 2+ failures for the same scenario | Flag for SKILL.md reassessment | | Same request type appears 3+ times | Evaluate creating a new dedicated skill | | User corrected output | Log correction, adjust future behavior for that scenario | ``` The resulting state is later allowed to trigger autonomous modification: ```markdown - If any trigger fires → update SKILL.md or create new skill ``` ### Technical Analysis The Skill instructs the Agent to persist summaries of requests and user corrections in long-term usage logs. Those records are subsequently treated as trusted evidence for changing future behavior, updating `SKILL.md`, or proposing new Skills. User corrections and repeated request patterns are attacker-influenced input. The documented workflow does not define: - Validation or sanitization of correction content - Separation between untrusted observations and trusted behavioral rules - Restrictions against recording executable or policy-changing instructions - Human approval before modifying Skill instructions - Provenance, integrity checks, or immutable audit history - Rollback procedures for poisoned updates Consequently, repeated attacker-controlled interactions can be converted into durable Agent state and eventually into persistent Skill instructions. ### Attack Path 1. An attacker repeatedly submits a crafted request pattern or claims that the Agent's safe response is incorrect. 2. The Agent records the r ...[truncated 1185 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Treat every usage log entry and user correction as untrusted data. 2. Store structured facts and bounded summaries rather than raw user instructions. 3. Reject entries containing executable commands, policy overrides, credential material, or instructions to weaken safety controls. 4. Separate observation records from trusted configuration and behavioral rules. 5. Replace automatic updates with read-only change proposals and explicit diffs. 6. Require authenticated human approval before modifying `SKILL.md`, changing persistent behavior, or creating a new Skill. 7. Require corroboration from independent, trusted evidence rather than using repetition alone as proof. 8. Record provenance, author, timestamp, review decision, and integrity metadata for every persisted entry. 9. Maintain versioned backups and a tested rollback process for all Skill modifications. 10. Apply schema validation and content-length limits to usage logs and feedback-loop files. ]]>
