T02 · Agent Memory Poisoning
- Location
scripts/brain.py:412- Finding
Untrusted Learning Content Can Be Persistently Injected into Agent Control and Memory Files
- Content
View full analysis
- Remediation
View remediation
Security audit
Security checks for vulnerabilities and agentic risk
This skill is a self-improving memory tool, but it can persistently change future agent behavior without enough review or reliable rollback.
Install only if you are comfortable with a skill that keeps long-term local memory and can alter future agent guidance. Do not run evolve on unreviewed learnings, do not store secrets in learning or error text, and treat rollback as incomplete unless the workspace files and ~/.adaptive-brain state are manually reviewed.
scripts/brain.py:412Untrusted Learning Content Can Be Persistently Injected into Agent Control and Memory Files
scripts/brain.py:548Rollback Reports Success Without Reverting Promoted Workspace Rules or Behavioral State
The evolution mechanism creates a trust-escalation pipeline: untrusted learned text is stored, confidence-adjusted, marked promotable, and then written into durable workspace instruction files and DNA state. In an agent skill context, this is especially dangerous because seemingly harmless text can become authoritative future behavior, enabling prompt/instruction persistence and long-term policy poisoning.
The rollback command claims to restore a previous evolution state, but it only truncates the evolution history log and does not revert changes to brain.json, learnings.json, DNA state, or appended workspace markdown files. This creates a deceptive recovery mechanism: operators may rely on rollback for safety, auditability, or containment, yet promoted instructions and persistent state remain active after 'rollback'.
The trigger list includes very broad phrases such as "improve yourself" and generic correction/error conditions, which can cause the skill to activate in contexts far beyond explicit user intent. Because this skill can adapt behavior and modify persistent state, ambiguous activation materially increases the risk of unintended autonomous changes.
The skill describes adaptation behavior without a prominent warning that it writes persistent files and may promote changes into workspace guidance files. Users may invoke it expecting analytics or logging, while the skill actually performs self-modifying and cross-file updates that can alter future agent behavior.
The skill explicitly proposes turning accumulated learnings into permanent rules and generating SKILL.md patches, which is a form of persistent context injection. This is dangerous because incorrect, adversarial, or low-quality inputs can be fossilized into future agent instructions, causing long-lived behavior manipulation across sessions.
The brain reviews all learnings and:
1. Identifies patterns that should become permanent rules
2. Generates optimized SKILL.md patches
3. Creates behavioral DNA mutations
4. Tracks evolution history (for rollback)
The scheduled adaptation section encourages unattended recurring execution of a system that can automatically modify persistent and workspace files, but it does not warn about the autonomy or scope of those modifications. This creates a durable mechanism for silent drift in agent behavior and persistent context over time, even without fresh user review.
The learn command persistently stores raw user-supplied summary, context, and fix text, then later surfaces or promotes that text in listings, reports, DNA entries, and workspace files. If users include secrets, internal URLs, credentials, prompts, or sensitive operational details, the system preserves and re-exposes them in plain language without redaction controls.
The error logging path stores raw command strings, error messages, fix text, and file lists, all of which commonly contain secrets such as API keys, tokens, internal paths, environment values, or customer data. That data is then reused for related-learning detection and may later be printed or promoted, turning transient secret exposure into persistent leakage.
The evolve flow automatically appends learned rules into persistent workspace instruction files without approval, review, or safety filtering. Because the learned text originates from prior user-provided summaries and fixes, this creates an instruction-injection path where untrusted content can become durable behavioral guidance for future agent runs.
The rollback command writes an irreversible truncation of the evolution history to disk immediately, without any confirmation prompt or dry-run option. This is a safety weakness because a mistaken invocation can destroy forensic or audit history and impair incident review.