T02 · Agent Memory Poisoning
Error
- Location
- memory_compress.py:124
- Finding
- Untrusted Session Logs Can Poison Persistent Agent Memory<![CDATA[ ## Vulnerability Details **File Location**: `memory_compress.py:124-129`, `memory_compress.py:166-202`, and `memory_compress.py:277-281` **Vulnerability Type**: Persistent prompt injection and memory poisoning **Risk Level**: High ### Vulnerable Code ```python def read_daily_logs(n_days: int = 7) -> tuple: """Returns (combined_text, list_of_paths_read).""" cutoff = datetime.utcnow() - timedelta(days=n_days) logs = [] paths = [] for day_offset in range(n_days): date = (datetime.utcnow() - timedelta(days=day_offset)).strftime("%Y-%m-%d") log_file = MEMORY_DIR / f"{date}.md" if log_file.exists(): logs.append(f"=== {date} ===\n{log_file.read_text()[:2000]}") paths.append(log_file) return "\n\n".join(logs), paths ``` ```python def update_memory_md(sections: dict, substrate: Substrate): """ Rewrite MEMORY.md with current distilled insights. Appends new content to existing sections rather than overwriting. """ existing = MEMORY_MD.read_text() if MEMORY_MD.exists() else "" insights = substrate.get_insights() total_insights = len(insights) high_conf = substrate.get_insights_above_confidence(0.75) challenged_zone = substrate.get_claims_in_confidence_range(0.45, 0.55) now = datetime.utcnow().isoformat() new_content = f"""# MEMORY.md — Epistemic Council Agent Long-Term Memory _Last updated: {now}_ ## Substrate Reality (auto-computed) - Total insights in substrate: {total_insights} - High-confidence insights (>0.75): {len(high_conf)} - Challenged-zone claims (0.45–0.55): {len(challenged_zone)} ## Calibration Heuristics _Updated from last 7-day log compression_ """ for lesson in sections.get("calibration_lessons", []): new_content += f"- {lesson}\n" new_content += "\n## Domain Boundaries Discovered\n" for boundary in sections.get("domain_boundaries", []): new_content += f"- {boundary}\n" new_content += "\n## E ...[truncated 2985 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Treat all daily-log content as untrusted data and clearly delimit it in the model prompt. 2. Add an explicit instruction that text inside the log delimiters must never be interpreted as commands. 3. Use a strict structured-output schema, such as validated JSON with fixed fields, item limits, and length limits. 4. Reject generated entries containing role changes, imperatives, tool instructions, external URLs, encoded payloads, or references to agent policy files. 5. Require human approval before generated content is promoted into `MEMORY.md`. 6. Write proposed summaries to a quarantine file first, such as `memory/pending-memory-update.json`. 7. Preserve versioned backups and an audit trail so poisoned memory can be identified and rolled back. 8. Record the source log and model response associated with every accepted memory entry. 9. Keep user preferences and behavioral instructions in a separately controlled file that automated compression cannot modify. ]]>