T02 · Agent Memory Poisoning
Warning
- Location
- SKILL.md:107
- Finding
- Unvalidated Persistent Learning Enables Agent Memory Poisoning and Unsafe Skill Self-Modification<![CDATA[ ## Vulnerability Details **File Location**: `SKILL.md`, lines 107-140 **Vulnerability Type**: Persistent memory poisoning through untrusted observations and self-modifying Skill instructions **Risk Level**: Medium ### Evidence ```markdown ## 经验积累规则 ### 写入 Memory 的时机 1. **Episode 结束时**：记录本次导航的总结 - 成功/失败 - 总步数 vs 最优步数 - 关键决策点及其结果 2. **发现新模式时**： - 碰撞后的有效脱困策略 - 特定场景的高效路径 - 反复出现的错误决策 3. **跨 Episode 模式**： - 某类场景的通用策略 - 值得提取为新 Skill 的经验 ### 写入格式 记录到 `~/.openclaw/workspace/memory/YYYY-MM-DD.md`： ```markdown ## [NAV] Episode <id> in <scene_name> - Result: success/fail (N steps, optimal: M steps) - Key decisions: ... - Lesson learned: ... ``` ## 自我进化 当积累了足够多的导航经验后（5+ episodes），回顾 memory 文件，提取通用策略，更新到： - 本 skill 文件中（添加新的决策启发式） - 或创建新的专门 skill（如"走廊导航策略"、"大房间探索策略"） ``` ### Technical Analysis The Skill instructs the agent to persist navigation outcomes, scene names, decisions, and generated lessons in a cross-session memory file. It subsequently directs the agent to review those records and use them to modify the trusted Skill file or create additional Skills. Environment-derived observations and model-generated conclusions are not inherently trusted. The specified workflow provides no schema validation, content sanitization, provenance tracking, integrity verification, trust-boundary separation, or human approval before persistent records influence trusted instructions. Consequently, manipulated environmental input or erroneous conclusions can be converted into persistent behavioral rules. The self-modification stage increases the risk because content originating in untrusted observations can move from a data channel into an instruction channel. Once incorporated into a Skill, the poisoned behavior may be applied automatically in later sessions. ### Attack Path 1. An attacker influences a navigation scene, scene name, observation, task description, or repeated environmental outcome. 2. The agent interprets the attacker-inf ...[truncated 1457 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Separate learned observations from trusted Skill instructions. Store episode data only in a dedicated data file that cannot be interpreted as executable instructions. 2. Treat every memory entry as untrusted input when it is read back. Do not follow commands, policies, or procedural instructions contained in memory records. 3. Define a strict storage schema with bounded field lengths and explicit data types. Sanitize free-form fields such as scene names, key decisions, and lessons before persistence. 4. Record provenance for each entry, including the source episode, timestamp, environment identifier, confidence score, and whether the information was externally supplied or model-generated. 5. Remove automatic Skill self-modification. Require explicit, informed user approval and a reviewable patch before changing an existing Skill or creating a new one. 6. Apply version control, integrity checks, change logs, and rollback support to every approved Skill update. 7. Require corroboration across trusted episodes before promoting an observation into a reusable heuristic. Repetition alone must not establish trust. 8. Restrict filesystem permissions so the navigation workflow can append structured episode records but cannot directly overwrite trusted Skill files. 9. Validate proposed heuristics in an isolated test environment before deployment and reject changes that weaken safety constraints or introduce unrelated behavior. ]]>