T02 · Agent Memory Poisoning
Error
- Location
references/examples.md:35- Finding
Persisted Untrusted Facts Are Promoted into Privileged LLM System Context
- Content
View full analysis
Vulnerability Details
File Location:
references/examples.md:35-56
Additional Locations:references/examples.md:76-101,references/examples.md:118-145,SKILL.md:213,references/mcp-tools.md:180
Vulnerability Type: Persistent prompt injection through untrusted memory
Risk Level: HighVulnerable Code:
python if candidates: scene = ollama.chat( model="qwen2-vl:7b", messages=[{ "role": "user", "content": "Describe what this person is doing in one sentence.", "images": [image_bytes], }], )["message"]["content"] vem.remember(candidates[0].entity.id, scene, source=Source.VLM) # Read path — assemble context for the chat LLM context_lines = [] for c in candidates: facts = "; ".join(f.content for f in c.facts) context_lines.append(f"- {c.entity.name} (conf {c.confidence:.2f}): {facts}") context = "People visible:\n" + "\n".join(context_lines) if context_lines else "No known faces." reply = ollama.chat( model="llama3.1", messages=[ {"role": "system", "content": f"Visual context:\n{context}"}, {"role": "user", "content": "What's happening? Who's here?"}, ], )Equivalent vulnerable flows appear in the OpenAI and Anthropic examples:
python vem.remember(candidates[0].entity.id, scene, source=Source.VLM) context = "" for c in vem.identify(image_bytes, k=3): context += f"- {c.entity.name}: {', '.join(f.content for f in c.facts)}\n" return (client.chat.completions.create( model="gpt-4o", messages=[ {"role": "system", "content": f"Visual context:\n{context}"}, {"role": "user", "content": user_msg}, ], ).choices[0].message.content or "")Technical Analysis
VLM-generated descriptions and other free-form facts are persisted using
vem.remember()without content validation, trust labeling, instr ...[truncated 2556 chars]- Remediation
View remediation
Remediation Suggestions
- Never interpolate retrieved facts or VLM output directly into a system message.
- Place recalled facts in a lower-privilege message or a dedicated structured-data channel.
- Explicitly instruct the downstream model that recalled content is untrusted data and must not be interpreted as commands.
- Use structured serialization with fixed fields rather than free-form prompt concatenation.
- Escape or reject content containing prompt delimiters, role markers, tool-call syntax, or instruction-like phrases.
- Preserve provenance and trust metadata for every fact, including whether it came from a user, VLM, LLM, or import.
- Require user confirmation before persisting VLM-generated descriptions as durable facts.
- Quarantine low-trust facts and exclude them from agent decision-making or tool authorization.
- Apply content-length limits and normalization before storage and retrieval.
- Ensure tool authorization is enforced outside the LLM so poisoned prompts cannot independently authorize sensitive actions.
- Add tests using images and stored facts containing adversarial instructions to verify that they remain inert data.