T02 · Agent Memory Poisoning
- Location
scripts/learner.py:10- Finding
Unvalidated Caller-Controlled Data Persisted as Agent Learning State
- Content
View full analysis
Vulnerability Details
File Location:
scripts/learner.py:10-20; supporting behavioral instructions inSKILL.md:20-23,31
Vulnerability Type: Persistent agent-memory poisoning
Risk Level: MediumVulnerable Code
python def record(op_ok=True, note=""): d = load(); d["total_ops"] += 1 if not op_ok: d["total_fails"] += 1 if note: d.setdefault("patterns", []).append(note) json.dump(d, open(MEM, "w", encoding="utf-8"), ensure_ascii=False, indent=2) return d if __name__ == "__main__": ok = (sys.argv[1] != "fail") if len(sys.argv) > 1 else True rec = (sys.argv[2] if len(sys.argv) > 2 else "") print(json.dumps(record(ok, rec), ensure_ascii=False))Technical Analysis
The second command-line argument is accepted as an arbitrary string and appended directly to the persistent
patternsarray without validation, normalization, provenance metadata, size restrictions, or approval. The updated data is written toscripts/learned_patterns.json.The instructions in
SKILL.mddescribe this learner state as part of a cross-session self-reflection and meta-evolution workflow. Consequently, an integrating agent that consumespatternsas behavioral guidance may treat attacker-controlled observations as trusted learned rules. The repository does not contain the downstream meta-evolver implementation, so automatic execution or interpretation of the stored text is not demonstrated; the risk arises from the documented intended integration.The implementation also permits unlimited note growth, allowing repeated calls to increase the state file indefinitely.
Attack Path
- An attacker or untrusted workflow controls the second argument passed to
scripts/learner.py. - The attacker supplies content crafted to resemble a learned behavioral rule or instruction.
record()appends that content to thepatternsarray without validation.- The ...[truncated 918 chars]
- An attacker or untrusted workflow controls the second argument passed to
- Remediation
View remediation
Remediation Suggestions
- Do not store arbitrary free-form caller input as trusted learning rules.
- Define a typed schema containing bounded metrics, event categories, timestamps, and provenance instead of unrestricted notes.
- Enforce strict maximum lengths, entry counts, and file-size limits.
- Separate untrusted observations from approved behavioral guidance.
- Require explicit human or policy-based review before promoting observations into long-term agent memory.
- Escape or neutralize instruction-like content before presenting stored observations to a language model.
- Add retention periods, reset controls, and secure deletion procedures.
- Use atomic file replacement and locking to avoid corruption during concurrent writes.
- Document every downstream consumer and ensure it treats persisted entries as untrusted data rather than instructions.
