T02 · Agent Memory Poisoning
- Location
capture.ts:57- Finding
Mixed-message filtering permits persistent memory poisoning
- Content
View full analysis
stripRecallMarkers(m.text)); if (!cleanedUser.some(isCapturableMessage)) { api.logger.info( `memory-core-plus: capture skipped (no capturable user messages out of ${cleanedUser.length})`, ); return; } // Proceed with full conversation extraction (user + assistant) const recent = extractMessagesOfRole(messages, ["user", "assistant"], cfg.autoCaptureMaxMessages); if (recent.length === 0) return; const cleaned = recent.map((m) => `${m.role}: ${stripRecallMarkers(m.text)}`); const conversationBlock = cleaned.join("\n\n"); if (conversationBlock.length < 20) { api.logger.info("memory-core-plus: capture skipped (conversation too short)"); return; } const dateStr = formatDateStamp(); const sessionKey = `:memory-capture:${ctx.agentId ?? "default"}`; const captureStart = Date.now(); try { const result = await api.runtime.subagent.run({ sessionKey, message: buildCapturePrompt(conversationBlock, dateStr), extraSystemPrompt: CAPTURE_SYSTEM_PROMPT, idempotencyKey: randomUUID(), }); ``` ### Technical Analysis The capture eligibility check uses: ```ts cleanedUser.some(isCapturableMessage) ``` This only establishes that at least one recent user message is considered safe. It does not remove messages that fail `isCapturableMessage()` from the conversation subsequently supplied to the memory-extraction subagent. After this check succeeds, the code independently reconstructs the complete recent conversation, including both user and assistant messages. Every extract ...[truncated 3310 chars]- Remediation
View remediation
({ role: m.role, text: stripRecallMarkers(m.text), })) .filter((m) => isCapturableMessage(m.text)) .map((m) => `${m.role}: ${m.text}`); if (cleaned.length === 0) return; ``` 2. **Apply stricter trust rules by role.** Treat user and assistant content as untrusted. Assistant messages may repeat attacker input or contain model-generated instructions and should not bypass validation. 3. **Separate extraction from persistence.** Run the model without general workspace-writing tools, require structured output such as a validated JSON array of candidate facts, and let trusted plugin code perform the append operation. 4. **Validate provenance.** Persist only facts directly supported by eligible source messages. Reject instruction-like, policy-like, executable, or role-changing content even if the extraction model labels it as a fact. 5. **Add an explicit confirmation option.** For deployments handling untrusted users or shared workspaces, require user approval before committing new long-term memories. Automatic capture should be opt-in where persistent storage has security or privacy consequences. 6. **Constrain stored content.** Enforce maximum lengths, allowed data shapes, permitted headings, and rules preventing stored entries from resembling system or tool instructions. 7. **Strengthen recall isolation.** Continue marking memories as untrusted, but use a structured data channel where supported instead of concatenating memory text into natural-language prompt context. 8. **Add regression tests for mixed conversations.** Tests should verif ...[truncated 314 chars]
