T02 · Agent Memory Poisoning
- Location
scripts/learner.py:12- Finding
Unvalidated Notes Can Poison Persistent Agent Learning State
- Content
View full analysis
Vulnerability Details
File Location:
scripts/learner.py:12-16
Related Instructions:SKILL.md:19,SKILL.md:22-24,SKILL.md:34
Vulnerability Type: Persistent storage of untrusted agent-learning content
Risk Level: MediumVulnerable Code
python def record(op_ok=True, note=""): d = load(); d["total_ops"] += 1 if not op_ok: d["total_fails"] += 1 if note: d.setdefault("patterns", []).append(note) json.dump(d, open(MEM, "w", encoding="utf-8"), ensure_ascii=False, indent=2) return dThe command-line interface supplies the untrusted value directly to this function:
python rec = (sys.argv[2] if len(sys.argv) > 2 else "") print(json.dumps(record(ok, rec), ensure_ascii=False))Technical Analysis
The
record()function appends the suppliednoteverbatim to the persistentpatternsarray inscripts/learned_patterns.json. It applies no schema validation, provenance tracking, trust classification, size limit, content filtering, or approval process.The skill documentation directs the agent to record failure patterns and inject the learner into a cross-session evolution loop. Consequently, task content, browser-derived content, or model-generated reflections may be promoted into persistent learning state. If a later learner or meta-evolver component interprets these entries as behavioral guidance, instruction-like attacker content can continue influencing the agent after the original task has ended.
The reviewed package does not contain a component that reads
patternsand executes it as code or instructions. Therefore, direct code execution is not established. The security risk arises from the documented intended reuse of this persistent state by external agent-learning components.Attack Path
- An attacker places adversarial instructions in content processed during a skill task, or otherwise influences the reflection note passed to
learner.py. - The agent or an orch ...[truncated 1133 chars]
- An attacker places adversarial instructions in content processed during a skill task, or otherwise influences the reflection note passed to
- Remediation
View remediation
Remediation Suggestions
- Replace free-form persistent notes with a strict structured schema containing fields such as event type, trusted source, timestamp, task identifier, and bounded evidence.
- Treat all task-derived, browser-derived, and model-generated text as untrusted observations rather than executable rules or behavioral instructions.
- Enforce maximum lengths, entry-count limits, and retention periods to prevent persistent storage exhaustion.
- Separate raw observations from approved learning rules. Require explicit human review or a trusted validation stage before promoting an observation into reusable behavioral memory.
- Reject or quarantine entries containing instruction-like directives, role changes, tool requests, or attempts to override system and safety policies.
- Ensure downstream components never concatenate stored notes into privileged prompts without clear untrusted-data delimiters and explicit instructions not to follow embedded directives.
- Record provenance and integrity metadata so downstream consumers can distinguish trusted system-generated rules from externally influenced content.
- Provide an administrative mechanism to inspect, revoke, and reset learned state.
- Use atomic file replacement and restrictive file permissions when writing the JSON state to reduce corruption and unauthorized local modification risks.
