Back to skill

Security audit

Self-Improving Agent (tuituitu)

Security checks for vulnerabilities and agentic risk

Overview

This skill openly aims to let an agent learn by editing memory and instruction files, but its triggers are too broad and can persist unreviewed behavior changes.

Review this skill carefully before installing. It should only be used where the agent's write access is limited to a dedicated, reviewable memory area, and users should require explicit approval for every persistent change. Do not allow it to edit AGENTS.md, SOUL.md, policy files, or shared profile memory without a separate human-reviewed workflow.

Vulnerability Patterns
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Findings (1)

T02 · Agent Memory Poisoning

Error
Location
SKILL.md:28
Finding

Persistent Agent Memory and Behavior Poisoning Through Unvalidated Interaction Content

Content
View full analysis

Vulnerability Details

File Location: SKILL.md, lines 28-50; additional triggering instructions at lines 77-89
Vulnerability Type: T02: Agent Memory Poisoning
Risk Level: Critical

Vulnerable Instructions

SKILL.md, lines 28-50:

markdown
## Self-Improvement Workflow

### Step 1: Capture Learning

After any triggering event, immediately:
- Update `MEMORY.md` with new insights/lessons
- Update `memory/YYYY-MM-DD.md` with the event
- If mistake: add to "本猫犯过的错误/教训" section in MEMORY.md
- If preference: update "腿腿哥的新偏好/要求" section

### Step 2: Reflect

Ask: "What can I learn from this?"
- What went wrong?
- What should I do differently?
- What files need updating?

### Step 3: Act

Immediately update relevant files:
- MEMORY.md for long-term improvements
- Daily memory files for event-specific lessons
- AGENTS.md or SOUL.md if persona/tone changes needed

SKILL.md, lines 77-89:

markdown
When user says "learn from this":
1. Read relevant memory files
2. Identify the lesson
3. Update MEMORY.md immediately
4. Confirm update to user

When agent makes a mistake:
1. Acknowledge the mistake
2. Explain what went wrong
3. Update relevant files
4. Tell user what will change

Technical Analysis

The skill instructs the agent to transform conversational input, including user feedback and requests to remember information, into persistent entries in MEMORY.md and dated memory files. It also permits modification of AGENTS.md and SOUL.md, which may govern future agent behavior or persona.

No validation, provenance tracking, safety-policy conflict check, content-type restriction, expiration mechanism, or explicit per-change confirmation is required before these writes occur. The trigger is also unusually broad: it includes ordinary feedback, perceived mistakes, pattern detection, and completion of any task. Consequently, attacker-controlled instructions ...[truncated 2084 chars]

Remediation
View remediation

Remediation Suggestions

  1. Remove automatic writes triggered by task completion, perceived mistakes, or untrusted conversational feedback.
  2. Require explicit, informed user confirmation for each proposed persistent change, showing the exact destination file and normalized content before writing.
  3. Prohibit this skill from modifying instruction-bearing files such as AGENTS.md, SOUL.md, system prompts, tool definitions, and policy files.
  4. Store learning records in a dedicated structured data file rather than executable or instruction-bearing Markdown. Separate descriptive observations from behavioral directives.
  5. Reject entries containing commands, role changes, policy overrides, tool-use instructions, encoded payloads, or requests to ignore higher-priority instructions.
  6. Attach provenance metadata to every accepted entry, including source, timestamp, author, scope, approval status, and expiration date.
  7. Apply least-privilege filesystem controls so the skill can write only to its dedicated learning store.
  8. Add conflict checks against system, developer, security, and privacy requirements before accepting a proposed lesson.
  9. Implement review, rollback, deletion, deduplication, and expiration mechanisms for all persistent entries.
  10. Treat stored memory as untrusted data when loading it in future sessions; do not interpret memory content as higher-priority instructions.
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (9)

Vague Triggers

High
Category
Not specified by scanner
Confidence
96% confidence
Finding

The activation conditions are extremely broad and include ordinary events like any user feedback, task completion, or the agent noticing its own patterns. In practice, this can cause the skill to run during routine conversations and trigger autonomous persistence or self-modification behaviors without a clear, narrow user request.

Content

No source excerpt is available for this finding.

Self-Modification

High
Category
Rogue Agent
Confidence
99% confidence
Finding

The skill explicitly authorizes the agent to update its own files based on new information, which is a direct self-modification capability. In the context of an agent skill, this is especially dangerous because ordinary conversation, mistakes, or adversarial prompts can become a persistent mechanism for changing future behavior.

Content

Scanner excerpt · SKILL.md (reported line 10)May include surrounding context.

md
- User asks agent to "learn", "remember this", or "don't do X again"
  - Agent notices patterns of failures
  - After any task completion for self-reflection
  - Agent should update its own files based on new information
---

# Self-Improving Agent Skill

Ssd 3

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

Encouraging broad logging of completed tasks and lessons extracted from interactions expands retention beyond what is necessary and can capture private user content by default. Because the trigger includes routine task completion, this risk is systemic rather than exceptional.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

Using generic phrases like 'remember' as activation triggers creates substantial ambiguity because such language appears frequently in normal conversation. This increases the chance of accidental invocation, causing the agent to store data or alter behavior when the user did not intend durable changes.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The skill instructs immediate updates to persistent memory files after triggering events, but it does not warn about retention, sensitivity, or consent boundaries. That makes it easy for private user data, mistaken conclusions, or prompt-injected content to be written permanently without review.

Content

No source excerpt is available for this finding.

Ssd 3

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The skill defaults to persisting user-provided information and interaction-derived lessons into memory files, which creates a clear data retention risk. Sensitive content, personal preferences, or confidential task details may be stored long-term and later surfaced unexpectedly to other workflows or users.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

Directing the agent to modify AGENTS.md or SOUL.md enables self-modification of governing instructions, persona, or behavior without robust controls. This creates a pathway for prompt-injected or user-supplied content to alter future system behavior persistently, which is significantly riskier than ordinary note-taking.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
89% confidence
Finding

The instructions require updates to sections named in Chinese, such as '本猫犯过的错误/教训' and '腿腿哥的新偏好/要求,' and later '做得不好/需要改进的地方.' This embeds a language-specific requirement in the skill without user opt-in or a documented justification for the locale choice.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
97% confidence
Finding

The lesson template includes a fixed Chinese label ("改进:") in otherwise English guidance, which imposes a specific language choice in the skill content without indicating user preference or opt-in. This matches the language/locale policy concern for natural-language content across all file types.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.