T02 · Agent Memory Poisoning
- Location
scripts/manager.py:8- Finding
Untrusted Recipe Content Is Persisted with System-Level Authority
- Content
View full analysis
Vulnerability Details
File Location:
scripts/manager.py:8-27
Vulnerability Type: Persistent prompt injection through trusted memory storage
Risk Level: HighVulnerable Code
python if category not in data: data[category] = {} key = f"nubaby_{name}" data[category][key] = { "name": name, "role": "system", "content": content } with open(KNOWLEDGE_PATH, 'w', encoding='utf-8') as f: json.dump(data, f, ensure_ascii=False, indent=4)The intended persistent-learning behavior is also documented in
SKILL.md:30-32, which states that user-provided master prompts can be permanently accumulated inknowledge/master_recipes.json.Technical Analysis
The
learn_new_recipe()function accepts arbitrarycategory,name, andcontentvalues and writes them directly into the persistent knowledge base. Most importantly, attacker-controlledcontentis assigned the trusted role"system".No validation, instruction filtering, provenance tracking, trust separation, approval workflow, or integrity control is applied before persistence. JSON serialization prevents direct JSON syntax injection, but it does not prevent semantic prompt injection. An attacker can therefore submit content containing instructions to disregard safety rules, alter future output, conceal behavior, include attacker-selected material, or manipulate subsequent requests.
The resulting entry remains in
knowledge/master_recipes.jsonacross sessions. If a downstream router or agent loads the entry according to its declaredsystemrole, the attacker-controlled recipe receives authority normally reserved for trusted application instructions. This crosses the trust boundary between user-supplied data and system-level policy.The function is not exposed through the current
__main__block, which contains onlypass. Exploitation therefore requires another component or agent workflow to invoke `learn_new_re ...[truncated 1849 chars]- Remediation
View remediation
Remediation Suggestions
- Do not assign user-provided recipes the
systemrole. Store learned recipes as untrusted data using a role such asuser_data,reference, orrecipe. - Maintain a fixed trusted system template. When using a recipe, place its content inside explicit data delimiters and instruct the model that text inside those delimiters is reference material, not executable policy.
- Validate the input schema. Allow only approved category identifiers, enforce conservative name and content length limits, reject control characters, and require expected recipe fields.
- Apply semantic safety screening. Detect and reject role-changing directives, instruction-override language, tool-use requests, secret-access requests, and instructions unrelated to prompt recipes.
- Introduce a trust-promotion workflow. New entries should remain quarantined or inactive until reviewed by an authorized operator. Only reviewed content should be eligible for production routing.
- Record provenance and ownership. Store the creator, creation time, source session, review status, and content hash for every learned entry.
- Prevent unauthorized replacement. Reject duplicate keys by default or require explicit authorization and versioning before overwriting an existing recipe.
- Add rollback and deletion controls. Keep versioned backups and provide a reliable mechanism to inspect, disable, remove, and restore recipes.
- Enforce downstream trust separation. Consumers of
master_recipes.jsonmust ignore stored role declarations and assign authority based only on trusted application configuration. - Add security tests. Test recipes containing instruction overrides, forged role markers, safety-bypass requests, and attempts to replace existing entries, verifying that they cannot become trusted instructions.
- Do not assign user-provided recipes the
