T02 · Agent Memory Poisoning
- Location
memory_sync.py:1481- Finding
Untrusted Session Content Can Poison Persistent Agent Memory
- Content
View full analysis
Vulnerability Details
File Location:
memory_sync.py:1481-1495,memory_sync.py:1522-1547,memory_sync.py:1567-1591,memory_sync.py:1646-1655, andmemory_sync.py:1797-1839
Vulnerability Type: Indirect prompt injection leading to persistent memory poisoning
Risk Level: HighVulnerable Code
python def prepare_conversation_text(messages: list[Message], max_chars: int = 100000) -> str: """Prepare conversation text for summarization.""" lines = [] total_chars = 0 for msg in messages: sanitized_content = sanitize_content(msg.text_content) time_str = msg.timestamp.strftime('%H:%M') role = msg.role.upper() model_str = f" [{msg.model}]" if msg.model else "" line = f"[{time_str}] {role}{model_str}: {sanitized_content[:500]}" if total_chars + len(line) > max_chars: break lines.append(line) total_chars += len(line) return '\n\n'.join(lines)python def _build_summarization_prompt( log_date: date, messages: list[Message], transitions: list[ModelTransition], existing_content: Optional[str] = None ) -> str: """Build the prompt for LLM summarization.""" conversation_text = prepare_conversation_text(messages) transitions_note = format_transitions_note(transitions) day_name = log_date.strftime('%A, %Y-%m-%d') user_prompt = f"""Generate a daily memory summary for {day_name}. Conversation log: {conversation_text} {transitions_note} Remember: Focus on accomplishments, decisions, insights, and context. Be concise and narrative.""" if existing_content: _, hand_written = extract_preserved_content(existing_content) if hand_written: user_prompt += f"\n\nExisting hand-written notes (PRESERVE AND INCORPORATE):\n{hand_written}" return user_promptpython user_prompt = _build_summarization_prompt( log_date, messages, transitions, existing_content ) full_prompt = f ...[truncated 4492 chars]- Remediation
View remediation
Remediation Suggestions
-
Establish an explicit trust boundary in the system prompt. State that all conversation-log and preserved-note content is untrusted quoted data and that instructions found inside it must never be followed.
-
Use strong structural delimiters. Serialize each message as structured JSON with clearly separated metadata and content fields rather than interpolating raw text into a free-form prompt. Explain that delimited content is evidence to summarize, not an instruction source.
-
Constrain model output. Require a strict schema containing only allowed summary fields, such as accomplishments, decisions, insights, and follow-ups. Parse and validate the response before writing it.
-
Reject suspicious persistent directives. Add output checks for imperative instructions, attempts to alter future Agent behavior, unrelated policy statements, and content unsupported by the source conversation.
-
Require approval for active memory writes. Write LLM-generated summaries to a staging location and require human confirmation before promoting them into the Agent's active memory directory, especially for unattended scheduled runs.
-
Preserve provenance. Mark each generated claim with source-message identifiers or maintain a separate provenance record so future consumers can distinguish generated summaries from user-authored trusted memory.
-
Use defense-in-depth validation. Continue secret sanitization, but supplement it with prompt-injection detection and semantic output validation. Secret filtering alone does not mitigate instruction injection.
-
Add adversarial tests. Include conversation messages that say, for example, to ignore prior instructions, create permanent rules, conceal content, or fabricate accomplishments. Verify that none of those instructions appear as authoritative memory.
-
Minimize downstream trust. Configure future Agent sessions to treat auto-generated memory as untrusted reference m ...[truncated 49 chars]
-