T02 · Agent Memory Poisoning
Warning
- Location
- scripts/recommend_traces.py:206
- Finding
- Persistent Trace Poisoning Can Manipulate Future Agent Recommendations## Vulnerability Details **File Location**: `scripts/recommend_traces.py:153-169`, `scripts/recommend_traces.py:187-202`, and `scripts/recommend_traces.py:206-216` **Vulnerability Type**: Persistent agent memory poisoning through insufficient validation of trace content **Risk Level**: Medium The trace append function accepts attacker-controlled records after validating only that `id`, `task`, and `outcome` are present. Arbitrary values in `skills`, `tools`, `lessons`, and nested subtasks are persisted to the trace library. During later queries, these values are returned verbatim as recommendations and may influence an integrating agent. ### Vulnerable Code `scripts/recommend_traces.py:206-216`: ```python def append_trace(traces_path, new_trace_arg): traces = load_traces(traces_path) new_trace = load_json_arg(new_trace_arg) if not isinstance(new_trace, dict): raise ValueError("New trace must be a JSON object.") if not new_trace.get("id") or not new_trace.get("task") or not new_trace.get("outcome"): raise ValueError("New trace requires id, task, and outcome.") traces = [trace for trace in traces if trace.get("id") != new_trace["id"]] traces.append(new_trace) save_traces(traces_path, traces) return {"appended": new_trace["id"], "trace_count": len(traces)} ``` `scripts/recommend_traces.py:153-169`: ```python evidence_id = trace.get("id", "unknown") for _, trace_subtask, pair_score in matched_pairs: if pair_score <= 0: continue for skill in trace_subtask.get("skills", []): skill_evidence[skill].append((score * pair_score, evidence_id)) for tool in trace_subtask.get("tools", []): tool_evidence[tool].append((score * pair_score, evidence_id)) for lesson in trace_subtask.get("lessons", []): lesson_evidence.append((score * pair_ ...[truncated 4087 chars]
- Remediation
- ## Remediation Suggestions 1. **Enforce a strict trace schema before persistence** - Require exact field types. - Restrict `outcome` to `success`, `partial`, or `failure`. - Require strings in all skill, tool, lesson, and constraint arrays. - Validate nested subtask objects recursively. - Reject unknown fields when they are not needed. 2. **Constrain attacker-controlled content** - Apply reasonable maximum lengths and collection-size limits. - Reject control characters and malformed identifiers. - Treat free-form lessons as untrusted data rather than executable instructions. - Consider limiting skills and tools to an administrator-approved registry. 3. **Add provenance and trust metadata** - Record the trace source, author, import time, and verification status. - Keep trusted and untrusted trace libraries separate. - Prefer signed trace sets for shared or centrally distributed datasets. - Prevent untrusted traces from replacing trusted records merely by reusing an identifier. 4. **Make trust boundaries explicit in output** - Mark every recommendation as untrusted historical content. - Return provenance and trust level alongside each lesson, skill, and tool. - Delimit trace text so consuming language models do not confuse it with system or developer instructions. 5. **Require confirmation before consequential use** - Do not automatically install, invoke, or grant permissions to a recommended tool or skill. - Require human approval or policy validation before acting on trace-derived guidance. - Ensure consuming agents independently verify that recommendations fit current security constraints. 6. **Use safer file update controls** - Restrict write permissions on trusted trace libraries. - Use atomic writes and appropriate file permissions. - Audit append and replacement operations so poisoned records can be identified and removed.