Back to skill

Security audit

Skill Self Evolution Enhancer

Security checks for vulnerabilities and agentic risk

Overview

This skill is not overtly malicious, but it gives other skills persistent self-modifying behavior with broad triggers and weak safeguards.

Install only if you intentionally want skills to gain persistent learning behavior. Use a dry run first, back up the target skill, review every generated file before use, avoid storing secrets or raw user data in .learnings, and require explicit approval before promoting learnings into durable rules.

Vulnerability Patterns
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (3)

T02 · Agent Memory Poisoning

Error
Location
SKILL.md:106
Finding
Untrusted User Feedback Can Poison Persistent Agent Behavior<![CDATA[ ## Vulnerability Details **File Location**: `SKILL.md:106-117`; `SKILL.md:128-145`; `SKILL.md:161-164`; `assets/EVOLUTION-RULES-TEMPLATE.md:15-23`; `assets/EVOLUTION-RULES-TEMPLATE.md:32-52`; `assets/LEARNINGS-TEMPLATE.md:21-49` **Vulnerability Type**: Persistent memory poisoning through untrusted conversation-derived rules **Risk Level**: High ### Vulnerable Code From `SKILL.md:106-117`: ```markdown The enhanced skill must **use** learnings, not only log them. Include this in EVOLUTION.md or the enhanced skill's instructions: ### Before Task - Load relevant entries from `.learnings/LEARNINGS.md` (and ERRORS.md if applicable) - Filter by area, tags, or keywords - Note which entries apply to the current task ### During Task - Apply learnings when relevant - Optionally annotate output: "本次参考了 [LRN-xxx]: ..." (or equivalent in target language) ``` From `assets/EVOLUTION-RULES-TEMPLATE.md:15-23`: ```markdown | Pattern proven across 3+ instances | Promote to {PROMOTION_TARGET_1} | | Safety/correctness rule | Promote to {PROMOTION_TARGET_2} | ## Review → Apply → Report Loop **Before task**: Load relevant entries from `.learnings/LEARNINGS.md` (and ERRORS.md if applicable). Filter by area, tags, or keywords. **During task**: Apply learnings when relevant. Optionally annotate: "本次参考了 [LRN-xxx]: ..." (or equivalent in target language). **After task**: Tell user which learnings were used, what evolution result, what improvement. Decide per-use mention vs end-of-task summary based on context. ``` From `assets/LEARNINGS-TEMPLATE.md:21-49`: ```markdown ## Learning Entry Format Append entries in this format: ```markdown ## [LRN-YYYYMMDD-XXX] category **Logged**: ISO-8601 timestamp **Priority**: low | medium | high | critical **Status**: pending **Area**: {DOMAIN_AREAS_EXAMPLE} ### Summary One-line description of what was learned ### Details Full context: what happened, what was wrong, what's correct ### Suggested Action Specific fix or improvem ...[truncated 2524 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Treat every conversation-derived learning as untrusted data rather than executable guidance. 2. Store the original feedback separately from normalized rules and prevent raw feedback from being loaded directly into the Agent instruction context. 3. Reject or quarantine entries containing instruction-control language, requests to ignore policies, tool directives, credential requests, or unrelated behavioral changes. 4. Bind each entry to its originating user, tenant, workspace, and Skill; do not apply one user's preferences to another user. 5. Require explicit human approval before changing an entry to `promoted` or writing it into an evolution, safety, or best-practice rule file. 6. Validate proposed rules against higher-priority safety policies before promotion and again every time they are loaded. 7. Replace the recurrence-count-only promotion condition with a review process based on trusted provenance, domain relevance, and independent validation. 8. Add an audit trail recording who created, reviewed, modified, approved, and applied each learning. 9. Provide rollback and deletion mechanisms for poisoned or obsolete entries. 10. Limit loaded entries to narrowly structured fields and escape or delimit all user-controlled text so it cannot be interpreted as Agent instructions. ]]>

T09 · Insecure Skill Coding Practices

Warning
Location
assets/ERRORS-TEMPLATE.md:20
Finding
Error Logging Template Can Persist Sensitive Diagnostic Data in Plaintext<![CDATA[ ## Vulnerability Details **File Location**: `assets/ERRORS-TEMPLATE.md:20-31` **Vulnerability Type**: Unredacted persistent logging of errors, inputs, parameters, and environment details **Risk Level**: Medium ### Vulnerable Code ```markdown ### Summary Brief description of what failed ### Error ``` Actual error message or output ``` ### Context - Operation attempted - Input or parameters used - Environment details if relevant ``` ### Technical Analysis The error template instructs enhanced Skills to persist raw error messages, command or operation output, input parameters, and environment details in `.learnings/ERRORS.md`. It provides no requirements for secret detection, redaction, data minimization, file permissions, or retention. Errors and parameters commonly include sensitive values such as API tokens, authorization headers, connection strings, personal information, internal hostnames, filesystem paths, and confidential user inputs. Persisting these values in a Markdown file expands their lifetime and makes them available to future Agent sessions and any process or user that can read the target Skill directory. ### Attack Path 1. A user or Agent invokes an operation containing a secret or confidential parameter. 2. The operation fails and includes that value in its error message, output, parameters, or environment details. 3. The enhanced Skill follows the template and appends the raw data to `.learnings/ERRORS.md`. 4. The log remains in the target Skill directory after the original task ends. 5. A later Agent session, collaborator, backup process, repository commit, or other local reader accesses the persistent plaintext value. An attacker who can deliberately trigger errors may increase the likelihood that sensitive runtime values are captured in the log. ### Impact Assessment The vulnerability does not itself bypass filesystem permissions. Exploitation requires the ability to trigger logging and subsequent access to the generated ...[truncated 519 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Add an explicit prohibition against recording passwords, API keys, tokens, cookies, authorization headers, private keys, connection strings, or complete environment-variable dumps. 2. Redact sensitive values before persistence using both key-name detection and value-pattern detection. 3. Record only an allowlisted subset of diagnostic fields required for troubleshooting. 4. Replace raw inputs and parameters with sanitized summaries, types, lengths, or irreversible hashes where appropriate. 5. Truncate error output and remove control sequences, embedded instructions, and confidential payloads. 6. Set restrictive permissions on generated learning files, such as owner-only access where supported. 7. Prevent `.learnings/` from being committed by default when it may contain operational data. 8. Define retention limits and provide secure deletion or automated expiration for old error records. 9. Warn users before recording diagnostic information that may contain confidential data. 10. Add tests covering redaction of common token, credential, URL, header, and environment-variable formats. ]]>

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/generate-evolution.sh:130
Finding
Scaffold Generator Overwrites Existing Files and Follows Destination Symlinks<![CDATA[ ## Vulnerability Details **File Location**: `scripts/generate-evolution.sh:130-144` **Vulnerability Type**: Unconditional file replacement and unsafe destination handling **Risk Level**: Medium ### Vulnerable Code ```bash # Create .learnings/ log_info "Creating $LEARNINGS_DIR/" mkdir -p "$LEARNINGS_DIR" # Copy templates with generic placeholders (agent fills domain-specific values later) log_info "Creating LEARNINGS.md, ERRORS.md, FEATURE_REQUESTS.md" cp "$ASSETS_DIR/LEARNINGS-TEMPLATE.md" "$LEARNINGS_DIR/LEARNINGS.md" cp "$ASSETS_DIR/ERRORS-TEMPLATE.md" "$LEARNINGS_DIR/ERRORS.md" cp "$ASSETS_DIR/FEATURE_REQUESTS-TEMPLATE.md" "$LEARNINGS_DIR/FEATURE_REQUESTS.md" # Create DOMAIN-CONFIG-DRAFT.md for agent to fill SKILL_NAME=$(basename "$TARGET_PATH") sed "s/\[skill-name\]/$SKILL_NAME/g" "$ASSETS_DIR/DOMAIN-CONFIG-TEMPLATE.md" > "$TARGET_PATH/DOMAIN-CONFIG-DRAFT.md" log_info "Created DOMAIN-CONFIG-DRAFT.md (agent should fill this)" # Create EVOLUTION.md with placeholders cp "$ASSETS_DIR/EVOLUTION-RULES-TEMPLATE.md" "$TARGET_PATH/EVOLUTION.md" ``` ### Technical Analysis The generator writes fixed destination filenames without checking whether they already exist. The `cp` commands and shell redirection therefore replace existing learning history, evolution rules, and domain configuration. The script also does not reject symbolic links at the destination paths. If a target Skill directory is attacker-controlled, an attacker can prepare one of the generated destinations as a symbolic link to another file. The copy or redirection may then modify the linked file using the invoking user's permissions. Although the target path must contain `SKILL.md`, that validation does not establish that the target directory or output files are trusted, owned by the invoking user, or free of symbolic links. ### Attack Path #### Existing-file destruction 1. A target Skill already contains `.learnings/LEARNINGS.md`, `.learnings/ERRORS.md`, `.learnings/FEATURE_REQUES ...[truncated 1373 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Refuse to overwrite existing files by default. 2. Use `cp --no-clobber` where supported, or create destinations with exclusive-create semantics. 3. Add an explicit `--force` option for intentional replacement and require confirmation when used interactively. 4. Back up existing files before any authorized replacement. 5. Check every destination with `test -e` and `test -L`; abort if it already exists or is a symbolic link unless a reviewed policy explicitly permits it. 6. Verify that the target directory and `.learnings` directory are real directories rather than symbolic links. 7. Create output in a securely created temporary file within the destination directory, then atomically rename it after validation. 8. Validate directory ownership and permissions before writing into a user-supplied target. 9. Print a complete conflict report before making changes. 10. Add tests for existing files, dangling and non-dangling symbolic links, concurrent generator runs, partial failures, and backup restoration. ]]>
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Findings (18)

Vague Triggers

High
Confidence
97% confidence
Finding
The trigger extraction guidance includes generic fallback phrases like '不对', '不是这样', '改一下', which are common in ordinary conversation and likely to match unrelated feedback. This can cause persistent logging or evolution actions from ambiguous remarks, enabling accidental or adversarial poisoning of a skill's learnings and causing self-modifying behavior based on weak signals.

Missing User Warnings

Medium
Confidence
94% confidence
Finding
The README states that the skill will generate .learnings/ and EVOLUTION.md for a target skill, but it does not clearly warn that this means writing into the target skill's directory. Because this skill is specifically designed to modify other skills, the missing disclosure can mislead users about side effects and increase the risk of unintended persistent changes.

Vague Triggers

Medium
Confidence
93% confidence
Finding
The documented trigger phrase is broad enough to overlap with normal user requests, which can cause the skill to activate unexpectedly. In a skill that adds self-evolution and writes learning artifacts, accidental activation increases the chance of unintended file generation or modification in other skills' directories.

Vague Triggers

Medium
Confidence
92% confidence
Finding
The manifest description is broadly worded to apply self-evolution to 'any skill' across many scenarios, which can cause over-activation and unintended invocation on generic user requests. In a skill-routing system, broad activation increases the chance this skill modifies or augments other skills without sufficiently specific user intent, expanding attack surface and making downstream risky behaviors easier to trigger.

Natural-Language Policy Violations

Medium
Confidence
95% confidence
Finding
Forcing generated files into an inferred language without asking the user can mis-handle multilingual contexts and cause incorrect or opaque configuration artifacts. In a self-evolution skill, this matters because generated rules and learnings may later drive automated behavior, so a mistaken language inference can reduce user oversight and increase the chance of unnoticed misconfiguration.

Ssd 3

Medium
Confidence
90% confidence
Finding
The skill explicitly instructs enhanced skills to retain prior interaction content in .learnings files and to surface used learnings back to the user in outputs and summaries. This creates a persistence and disclosure risk: sensitive user-provided details, corrections, or preferences may be stored long-term and later echoed inappropriately, especially when propagated across tasks or surfaced automatically.

Natural-Language Policy Violations

Medium
Confidence
95% confidence
Finding
Line L29 specifies a fixed target language field limited to `zh | en`. This is a natural-language policy concern because it imposes a language constraint without stating that the user may choose freely or that the limitation is justified by a region-specific purpose.

Vague Triggers

Medium
Confidence
92% confidence
Finding
This markdown template asks authors to define when the agent should check learnings, but only provides generic placeholders for activation conditions and no requirement for concrete scope, exclusions, or negative examples. That can lead to overly broad hook activation once instantiated, causing unintended skill invocation in ordinary contexts.

Vague Triggers

Medium
Confidence
89% confidence
Finding
The activation condition 'After suggesting optimization steps' is ambiguous because it triggers on the agent's own prior behavior rather than a clear external event or explicit user signal. In a self-evolution enhancer, this can create automatic post-action activation loops and excessive learning capture, especially in operational or troubleshooting skills where optimization steps are routine.

Vague Triggers

Medium
Confidence
93% confidence
Finding
The guidance explicitly recommends generic fallback triggers such as '用户纠正' and '操作失败', which are likely to occur during normal interaction and can cause the self-evolution workflow to activate far more often than intended. In a skill that adds persistent learning and rule-promotion behavior, overbroad triggers increase the chance of logging benign conversation as learning events, causing prompt-scope creep and unintended behavioral drift across future runs.

Natural-Language Policy Violations

Medium
Confidence
88% confidence
Finding
Mandating output in the language inferred from SKILL.md removes user choice and can lead to responses or generated scaffolding in a language the current user did not request or understand. In this skill's context, the main risk is misconfiguration, operator confusion, and reduced transparency of learning artifacts rather than direct code execution or privilege escalation.

Natural-Language Policy Violations

Medium
Confidence
91% confidence
Finding
The markdown includes a prescribed example response in Chinese: "本次参考了经验 [LRN-xxx]: ...". Because the document presents this as the feedback wording to use and does not offer language choice or explain a locale-specific requirement, it creates a natural-language locale policy concern.

Natural-Language Policy Violations

Medium
Confidence
94% confidence
Finding
The EVOLUTION.md integration section provides a full example sentence in Chinese as the model wording for user disclosure. Without any accompanying instruction to match the user's language or an explanation that the skill is region-specific, this is a language-policy violation in natural-language guidance.

Session Persistence

Medium
Category
Rogue Agent
Content
cat << EOF
Usage: $(basename "$0") <target-skill-path> [options]

Create self-evolution scaffolding in a target skill. Output structure matches self-improving-agent:
  target-skill/.learnings/LEARNINGS.md, ERRORS.md, FEATURE_REQUESTS.md

Arguments:
Confidence
60% confidence
Finding
Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.

Natural-Language Policy Violations

Low
Confidence
82% confidence
Finding
The placeholder example on L47 uses Chinese labels as one of the suggested values, but the template does not state that language selection is optional or tied to a specific locale. This can be read as steering skill outputs toward a specific language without documented opt-in.

Natural-Language Policy Violations

Low
Confidence
81% confidence
Finding
The template provides a concrete annotation example in Chinese ("本次参考了 ...") and later Chinese-language trigger examples, without stating that language choice should follow user preference. In a reusable template, this can normalize a fixed-language response pattern that conflicts with language/locale choice expectations.

Vague Triggers

Low
Confidence
76% confidence
Finding
This markdown template includes placeholder guidance for domain descriptions and areas, but it does not specify any concrete activation phrases, boundaries, or negative examples for when a related skill should or should not be invoked. In a skill ecosystem, such broad domain descriptors can contribute to ambiguous invocation behavior if reused as trigger guidance.

Natural-Language Policy Violations

Low
Confidence
88% confidence
Finding
The placeholder examples for DOMAIN_AREAS and PROMOTION_TARGETS include Chinese text and Chinese-named files as exemplar values. In a reusable template, this can implicitly steer skill authors toward a fixed language/locale without documenting that language choice is optional or context-specific.

Static analysis

No suspicious patterns detected.