T02 · Agent Memory Poisoning
- Location
scripts/recommend_traces.py:206- Finding
Persistent Trace Poisoning Can Manipulate Future Agent Recommendations
- Content
View full analysis
Vulnerability Details
File Location:
scripts/recommend_traces.py:153-169,scripts/recommend_traces.py:187-202, andscripts/recommend_traces.py:206-216
Vulnerability Type: Persistent agent memory poisoning through insufficient validation of trace content
Risk Level: MediumThe trace append function accepts attacker-controlled records after validating only that
id,task, andoutcomeare present. Arbitrary values inskills,tools,lessons, and nested subtasks are persisted to the trace library. During later queries, these values are returned verbatim as recommendations and may influence an integrating agent.Vulnerable Code
scripts/recommend_traces.py:206-216:python def append_trace(traces_path, new_trace_arg): traces = load_traces(traces_path) new_trace = load_json_arg(new_trace_arg) if not isinstance(new_trace, dict): raise ValueError("New trace must be a JSON object.") if not new_trace.get("id") or not new_trace.get("task") or not new_trace.get("outcome"): raise ValueError("New trace requires id, task, and outcome.") traces = [trace for trace in traces if trace.get("id") != new_trace["id"]] traces.append(new_trace) save_traces(traces_path, traces) return {"appended": new_trace["id"], "trace_count": len(traces)}scripts/recommend_traces.py:153-169:python evidence_id = trace.get("id", "unknown") for _, trace_subtask, pair_score in matched_pairs: if pair_score <= 0: continue for skill in trace_subtask.get("skills", []): skill_evidence[skill].append((score * pair_score, evidence_id)) for tool in trace_subtask.get("tools", []): tool_evidence[tool].append((score * pair_score, evidence_id)) for lesson in trace_subtask.get("lessons", []): lesson_evidence.append((score * pair_ ...[truncated 4087 chars]- Remediation
View remediation
Remediation Suggestions
-
Enforce a strict trace schema before persistence
- Require exact field types.
- Restrict
outcometosuccess,partial, orfailure. - Require strings in all skill, tool, lesson, and constraint arrays.
- Validate nested subtask objects recursively.
- Reject unknown fields when they are not needed.
-
Constrain attacker-controlled content
- Apply reasonable maximum lengths and collection-size limits.
- Reject control characters and malformed identifiers.
- Treat free-form lessons as untrusted data rather than executable instructions.
- Consider limiting skills and tools to an administrator-approved registry.
-
Add provenance and trust metadata
- Record the trace source, author, import time, and verification status.
- Keep trusted and untrusted trace libraries separate.
- Prefer signed trace sets for shared or centrally distributed datasets.
- Prevent untrusted traces from replacing trusted records merely by reusing an identifier.
-
Make trust boundaries explicit in output
- Mark every recommendation as untrusted historical content.
- Return provenance and trust level alongside each lesson, skill, and tool.
- Delimit trace text so consuming language models do not confuse it with system or developer instructions.
-
Require confirmation before consequential use
- Do not automatically install, invoke, or grant permissions to a recommended tool or skill.
- Require human approval or policy validation before acting on trace-derived guidance.
- Ensure consuming agents independently verify that recommendations fit current security constraints.
-
Use safer file update controls
- Restrict write permissions on trusted trace libraries.
- Use atomic writes and appropriate file permissions.
- Audit append and replacement operations so poisoned records can be identified and removed.
-
