T02 · Agent Memory Poisoning
- Location
references/learning-loop.md:3- Finding
Persistent Prompt Poisoning Through Attacker-Influenced Call Transcripts
- Content
View full analysis
Vulnerability Details
File Location:
SKILL.md:90-114;references/learning-loop.md:3-10, 45-73
Vulnerability Type: T02: Agent Memory Poisoning
Risk Level: HighRelevant code snippets:
SKILL.md:107-114text ## Learning Loop ArchitecturePer-call: audit_persona → persona_audits table After N calls: meta-graph analyzes patterns → generates prompt patches Validation: test patched prompts against baseline Deploy: write patches to prompt_patches table (runtime override, no redeploy)
text See `references/learning-loop.md` for the meta-graph architecture.references/learning-loop.md:3-10text ## Data FlowLive Call → fetch_transcript → analyze_call → audit_persona → persona_audits (Supabase) ↓ prompt_evolution meta-graph (periodic) ↓ prompt_patches table → persona_engine reads at runtime
text references/learning-loop.md:45-73text ## Prompt Evolution Meta-Graphanalyze_corpus → identify_patterns → generate_refinements → validate → apply
text ### analyze_corpus Pull last N audits grouped by archetype. For each: avg words/turn, enthusiasm leak frequency, flagged quote patterns. ### identify_patterns Aggregate: "Owl personas average 22 words/turn (target: 8-12)" / "Lamb says 'I appreciate that' in 4/7 calls" ### generate_refinements Surgical prompt patches, not full rewrites:ADD to Water: "Never say 'I appreciate that' — too warm for someone who didn't ask for this." CHANGE Owl length guidance: "5-12 words until Phase 4."
text ### validate Run 3-5 simulated calls with patched prompts. Compare metrics against baseline. ### apply Write to prompt_patches table. persona_engine reads active ...[truncated 2603 chars]- Remediation
View remediation
Remediation Suggestions
- Treat transcripts, audit recommendations, flagged quotations, and generated refinements as untrusted data rather than executable instructions.
- Do not derive unrestricted prompt text directly from transcript contents. Generate patches through a strict schema that only permits predefined behavioral parameters and bounded values.
- Default every new patch to
active = falseand requirevalidated = trueplus an explicit approval state before runtime loading. - Require human review for all generated prompt changes, especially global patches where
archetypeis null. - Add security validation separate from quality testing. Reject patches that attempt to change system priorities, tool permissions, data-access rules, safety constraints, or instruction hierarchy.
- Preserve transcript content as quoted data and explicitly prohibit the evolution component from following instructions found inside calls or audit fields.
- Record provenance for each patch, including source audits, generator version, approver, validation results, tenant, timestamp, and cryptographic integrity metadata.
- Isolate patches by tenant, persona, and archetype to prevent cross-user or cross-persona contamination.
- Apply rate limits, minimum corpus diversity requirements, anomaly detection, and source weighting so one caller or coordinated group cannot dominate the learning corpus.
- Version all prompt configurations and provide immediate disablement and rollback capabilities.
- Load patches only through an allowlisted composition layer that cannot override higher-priority system or safety instructions.
- Add adversarial validation cases specifically testing prompt injection, repeated phrase planting, poisoned recommendations, and cross-session behavioral persistence.