T02 · Agent Memory Poisoning
Error
- Location
- references/learning-loop.md:3
- Finding
- Persistent Prompt Poisoning Through Attacker-Influenced Call Transcripts## Vulnerability Details **File Location**: `SKILL.md:90-114`; `references/learning-loop.md:3-10, 45-73` **Vulnerability Type**: T02: Agent Memory Poisoning **Risk Level**: High **Relevant code snippets:** `SKILL.md:107-114` ```text ## Learning Loop Architecture ``` Per-call: audit_persona → persona_audits table After N calls: meta-graph analyzes patterns → generates prompt patches Validation: test patched prompts against baseline Deploy: write patches to prompt_patches table (runtime override, no redeploy) ``` See `references/learning-loop.md` for the meta-graph architecture. ``` `references/learning-loop.md:3-10` ```text ## Data Flow ``` Live Call → fetch_transcript → analyze_call → audit_persona → persona_audits (Supabase) ↓ prompt_evolution meta-graph (periodic) ↓ prompt_patches table → persona_engine reads at runtime ``` ``` `references/learning-loop.md:45-73` ```text ## Prompt Evolution Meta-Graph ``` analyze_corpus → identify_patterns → generate_refinements → validate → apply ``` ### analyze_corpus Pull last N audits grouped by archetype. For each: avg words/turn, enthusiasm leak frequency, flagged quote patterns. ### identify_patterns Aggregate: "Owl personas average 22 words/turn (target: 8-12)" / "Lamb says 'I appreciate that' in 4/7 calls" ### generate_refinements Surgical prompt patches, not full rewrites: ``` ADD to Water: "Never say 'I appreciate that' — too warm for someone who didn't ask for this." CHANGE Owl length guidance: "5-12 words until Phase 4." ``` ### validate Run 3-5 simulated calls with patched prompts. Compare metrics against baseline. ### apply Write to prompt_patches table. persona_engine reads active ...[truncated 2603 chars]
- Remediation
- ## Remediation Suggestions 1. Treat transcripts, audit recommendations, flagged quotations, and generated refinements as untrusted data rather than executable instructions. 2. Do not derive unrestricted prompt text directly from transcript contents. Generate patches through a strict schema that only permits predefined behavioral parameters and bounded values. 3. Default every new patch to `active = false` and require `validated = true` plus an explicit approval state before runtime loading. 4. Require human review for all generated prompt changes, especially global patches where `archetype` is null. 5. Add security validation separate from quality testing. Reject patches that attempt to change system priorities, tool permissions, data-access rules, safety constraints, or instruction hierarchy. 6. Preserve transcript content as quoted data and explicitly prohibit the evolution component from following instructions found inside calls or audit fields. 7. Record provenance for each patch, including source audits, generator version, approver, validation results, tenant, timestamp, and cryptographic integrity metadata. 8. Isolate patches by tenant, persona, and archetype to prevent cross-user or cross-persona contamination. 9. Apply rate limits, minimum corpus diversity requirements, anomaly detection, and source weighting so one caller or coordinated group cannot dominate the learning corpus. 10. Version all prompt configurations and provide immediate disablement and rollback capabilities. 11. Load patches only through an allowlisted composition layer that cannot override higher-priority system or safety instructions. 12. Add adversarial validation cases specifically testing prompt injection, repeated phrase planting, poisoned recommendations, and cross-session behavioral persistence.