T02 · Agent Memory Poisoning
Warning
- Location
README.md:33- Finding
Untrusted Human Feedback Can Poison Persistent Agent Memory
- Content
View full analysis
Vulnerability Details
File Location:
SKILL.md:35-41,README.md:33-44, andVERIFICATION_PROTOCOL.md:17-29
Vulnerability Type:T02: Agent Memory Poisoning
Risk Level: MediumComplete Vulnerable Snippets
SKILL.md:35-41— English translation of the audited documentation:text Implements the Agent's SOUL five-law self-evolution cycle: 1. Known-versus-unknown separation 2. Files as memory: write all intermediate state to persistent files 3. Four-direction collision reasoning 4. Human-machine closed-loop injection: accept human falsification input, structurally absorb it, and update the world model 5. Confidence labeling and red-line protectionREADME.md:33-44— relevant API segment, with the example feedback translated into English:python result = cog.evolve( task="Analyze potential risks in this code", known=["code structure", "dependency relationships"], unknown=["runtime behavior", "concurrency scenarios"] ) # result.output — final conclusion # result.confidence — confidence score # result.collision_log — reasoning details # result.file_written — path of the written memory file cog.inject_human_feedback( "I found a concurrency bug in production; the reproduction steps are..." )VERIFICATION_PROTOCOL.md:17-29— English translation of the relevant persistence requirements:text File-memory verification: - After every evolve() call, is VERIFICATION_LOG.md written? - Is session recovery supported by reading the previous state from a file? - Passing criterion: 100% successful file writes Human-machine closed-loop verification: - Is inject_human_feedback(feedback: str) provided? - Is feedback written to a persistent file and does it affect the next evolve() output? - Passing criterion: output changes by at least 30% after feedback injectionTechnical Analysis
The documented design acc ...[truncated 2130 chars]
- Remediation
View remediation
Remediation Suggestions
- Treat all human feedback and restored memory as untrusted data, never as authoritative instructions.
- Store feedback in a data-only schema with explicit fields for source, timestamp, scope, confidence, and verification status.
- Separate system instructions, user feedback, observations, and derived conclusions into independently protected stores.
- Require explicit user or administrator approval before untrusted feedback is promoted into durable memory.
- Validate feedback against the active task and reject attempts to introduce executable commands, policy overrides, hidden instructions, or unrelated behavioral rules.
- Apply integrity protection to memory files and restrict their filesystem permissions according to least privilege.
- Scope memory by project, user, and session to prevent one tenant or task from influencing another.
- Add expiration, versioning, audit history, deletion, and rollback controls for all persistent entries.
- Avoid storing hidden reasoning, credentials, private data, or unnecessary intermediate state. Persist only the minimum structured information needed.
- On session recovery, present restored entries as untrusted historical context and require revalidation before they affect output.
- Add adversarial tests demonstrating that feedback containing instruction overrides cannot modify system policy or unrelated future tasks.
- Provide the actual implementation for review and ensure its behavior matches the documented validation and isolation controls.