T02 · Agent Memory Poisoning
Warning
- Location
scripts/learner.py:70- Finding
Persistent Untrusted Learning Data Can Influence Future Agent Behavior
- Content
View full analysis
Vulnerability Details
File Location:
scripts/learner.py:70-126, with the persistence workflow documented inSKILL.md:205-225
Vulnerability Type: Persistent memory poisoning through unsanitized user-controlled learning records
Risk Level: MediumVulnerable Code
python def record_op(skill_dir, capability, success=True, error=None, note=None): data = load_learning(skill_dir) data["totalOps"] += 1 cap = data["capabilityStats"].setdefault(capability, {"count": 0, "success": 0, "fail": 0}) cap["count"] += 1 if success: cap["success"] += 1 else: cap["fail"] += 1 data["totalErrors"] += 1 if error: ep = data["errorPatterns"].setdefault(error, {"count": 0, "lastNote": "", "lastTime": ""}) ep["count"] += 1 ep["lastNote"] = note or "" ep["lastTime"] = data["lastUpdated"] or datetime.datetime.now().strftime("%Y-%m-%dT%H:%M") rec = {"t": data["lastUpdated"], "cap": capability, "ok": success, "err": error, "note": note} data["recentOps"].append(rec) data["recentOps"] = data["recentOps"][-20:] save_learning(skill_dir, data) return data def record_preference(skill_dir, key, value): data = load_learning(skill_dir) data["preferences"][key] = value save_learning(skill_dir, data) return datapython def self_reflect(skill_dir): data = load_learning(skill_dir) suggestions = [] for err, info in data["errorPatterns"].items(): if info["count"] >= THRESHOLD_ERROR: suggestions.append( f"⚠️ Error “{err}” has occurred {info['count']} times; add validation or fallback steps, " f"and write the experience back into SKILL.md " f"(lastNote: {info.get('lastNote','')})" ) return suggestionsThe corresponding workflow in
SKILL.mddirects the agent to retai ...[truncated 2408 chars]- Remediation
View remediation
Remediation Suggestions
- Treat all persisted capability names, errors, notes, preference keys, and values as untrusted data.
- Define an allowlisted schema for preference keys and enforce type, character, and length restrictions on all stored fields.
- Reject or quarantine values containing instruction-like directives, role changes, tool commands, markup intended to alter prompts, or requests to bypass policy.
- Keep observations separate from executable or authoritative instructions. Never concatenate raw stored values into agent prompts without clear quoting and provenance labels.
- Remove the instruction to automatically rewrite
SKILL.md. Require an explicit, human-reviewed approval step for every proposed skill modification. - Require confirmation before applying a stored preference in a new session, especially when it changes tool use, security behavior, destination URLs, or output constraints.
- Store structured error identifiers rather than arbitrary error labels, and render free-form notes only as quoted diagnostic data.
- Add integrity controls and an audit trail recording who created, reviewed, approved, or deleted each persistent entry.
- Provide commands to inspect, reset, and selectively delete learned state before it affects future sessions.