T02 · Agent Memory Poisoning
Error
- Location
- SKILL.md:106
- Finding
- Untrusted User Feedback Can Poison Persistent Agent Behavior<![CDATA[ ## Vulnerability Details **File Location**: `SKILL.md:106-117`; `SKILL.md:128-145`; `SKILL.md:161-164`; `assets/EVOLUTION-RULES-TEMPLATE.md:15-23`; `assets/EVOLUTION-RULES-TEMPLATE.md:32-52`; `assets/LEARNINGS-TEMPLATE.md:21-49` **Vulnerability Type**: Persistent memory poisoning through untrusted conversation-derived rules **Risk Level**: High ### Vulnerable Code From `SKILL.md:106-117`: ```markdown The enhanced skill must **use** learnings, not only log them. Include this in EVOLUTION.md or the enhanced skill's instructions: ### Before Task - Load relevant entries from `.learnings/LEARNINGS.md` (and ERRORS.md if applicable) - Filter by area, tags, or keywords - Note which entries apply to the current task ### During Task - Apply learnings when relevant - Optionally annotate output: "本次参考了 [LRN-xxx]: ..." (or equivalent in target language) ``` From `assets/EVOLUTION-RULES-TEMPLATE.md:15-23`: ```markdown | Pattern proven across 3+ instances | Promote to {PROMOTION_TARGET_1} | | Safety/correctness rule | Promote to {PROMOTION_TARGET_2} | ## Review → Apply → Report Loop **Before task**: Load relevant entries from `.learnings/LEARNINGS.md` (and ERRORS.md if applicable). Filter by area, tags, or keywords. **During task**: Apply learnings when relevant. Optionally annotate: "本次参考了 [LRN-xxx]: ..." (or equivalent in target language). **After task**: Tell user which learnings were used, what evolution result, what improvement. Decide per-use mention vs end-of-task summary based on context. ``` From `assets/LEARNINGS-TEMPLATE.md:21-49`: ```markdown ## Learning Entry Format Append entries in this format: ```markdown ## [LRN-YYYYMMDD-XXX] category **Logged**: ISO-8601 timestamp **Priority**: low | medium | high | critical **Status**: pending **Area**: {DOMAIN_AREAS_EXAMPLE} ### Summary One-line description of what was learned ### Details Full context: what happened, what was wrong, what's correct ### Suggested Action Specific fix or improvem ...[truncated 2524 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Treat every conversation-derived learning as untrusted data rather than executable guidance. 2. Store the original feedback separately from normalized rules and prevent raw feedback from being loaded directly into the Agent instruction context. 3. Reject or quarantine entries containing instruction-control language, requests to ignore policies, tool directives, credential requests, or unrelated behavioral changes. 4. Bind each entry to its originating user, tenant, workspace, and Skill; do not apply one user's preferences to another user. 5. Require explicit human approval before changing an entry to `promoted` or writing it into an evolution, safety, or best-practice rule file. 6. Validate proposed rules against higher-priority safety policies before promotion and again every time they are loaded. 7. Replace the recurrence-count-only promotion condition with a review process based on trusted provenance, domain relevance, and independent validation. 8. Add an audit trail recording who created, reviewed, modified, approved, and applied each learning. 9. Provide rollback and deletion mechanisms for poisoned or obsolete entries. 10. Limit loaded entries to narrowly structured fields and escape or delimit all user-controlled text so it cannot be interpreted as Agent instructions. ]]>
