Back to skill

Security audit

Skill Evolve Pro

Security checks for vulnerabilities and agentic risk

Overview

This skill is a skill-optimizer, but it can send skill/session data to DeepSeek and persist model-generated changes to active skill files without enforcing the user confirmation its documentation promises.

Review this carefully before installing. Use only with non-sensitive skills and rollout/session data, rotate the exposed DeepSeek key, set your own API key through the environment, and require a manual diff review before allowing it to overwrite any active SKILL.md. Avoid running it against private SESSION-STATE.md or proprietary skills unless you have redaction and provider-approval controls in place.

Vulnerability Patterns
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (5)

T09 · Insecure Skill Coding Practices

Error
Location
scripts/config.py:12
Finding

Hardcoded DeepSeek API Credential Shipped in Source Code

Content
View full analysis
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Error
Location
scripts/evolve_core.py:145
Finding

Failure Trajectories and Answer Content Are Transmitted to an External LLM Without Redaction

Content
View full analysis
str: import urllib.request import urllib.error payload = { "model": MODEL, "messages": messages, "max_tokens": max_completion_tokens, "temperature": temperature, } for attempt in range(retry): try: req = urllib.request.Request( f"{DEEPSEEK_BASE_URL}/chat/completions", data=json.dumps(payload).encode("utf-8"), headers={ "Authorization": f"Bearer {DEEPSEEK_API_KEY}", "Content-Type": "application/json", }, method="POST", ) with urllib.request.urlopen(req, timeout=120) as resp: result = json.loads(resp.read().decode("utf-8")) return result["choices"][0]["message"]["content"] ``` The transmitted prompt is assembled from failure-trajectory fields: ```python for i, traj in enumerate(trajectories, 1): task = traj.get("task_description", traj.get("question", ""))[:300] fail_reason = traj.get("fail_reason", "unknown")[:300] predicted = traj.get("predicted_answer", "")[:200] reference = traj.get("reference_text", "")[:200] formatted.append( f"--- 轨迹 {i} ---\n" f"任务: {task}\n" f"失败原因: {fail_reason}\n" f"预测答案: {predicted}\n" f"参考答案: {reference}" ) ``` ### Technical Analysis The external LLM call is consistent with the skill's declared reflection functionality, so network access itself is functionally justified. However, the implementation sends use ...[truncated 2074 chars]
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/slow_update.py:101
Finding

Slow Update Sends Skill Document Content to an External Service Without Data Classification

Content
View full analysis
str: """调用 DeepSeek Chat API""" import urllib.request import urllib.error payload = { "model": MODEL, "messages": [ {"role": "system", "content": system}, {"role": "user", "content": user}, ], "temperature": 0.3, "max_tokens": max_tokens, } headers = { "Content-Type": "application/json", "Authorization": f"Bearer {DEEPSEEK_API_KEY}", } req = urllib.request.Request( DEEPSEEK_BASE_URL + "/chat/completions", data=json.dumps(payload).encode("utf-8"), headers=headers, method="POST", ) for attempt in range(3): try: with urllib.request.urlopen(req, timeout=60) as resp: result = json.loads(resp.read().decode("utf-8")) return result["choices"][0]["message"]["content"] ``` The request contains up to 6,000 characters of the current skill: ```python user_prompt = SLOW_UPDATE_USER_PROMPT.format( skill_content=current_skill[:6000], prev_slow_update=prev_slow_text, epoch=epoch, round=round_num, trajectory_stats=trajectory_stats_text, edit_history=edit_history_text, ) try: raw_response = _call_deepseek( system=SYSTEM_PROMPT, user=user_prompt, max_tokens=4096, ) ``` ### Technical Analysis Generating long-term guidance through an LLM is part of the declared functionality, making some external transmission necessary when this provider-backed mode is selected. Nevertheless, the function sends a large portion of the raw skill document, prior long-term guidance, trajectory statistics ...[truncated 1459 chars]
Remediation
View remediation

T02 · Agent Memory Poisoning

Error
Location
scripts/skill_scheduler.py:477
Finding

Remote Model Output Can Be Persisted Into Skill Instructions Without Required User Confirmation

Content
View full analysis
0: out_dir = args.skill_dir or robot_evolve_dir out_file = out_dir / "SKILL.md" out_file.write_text(result.skill_after, encoding="utf-8") print("") print(f"[scheduler] SKILL.md updated: {out_file}") else: print("") print("[scheduler] SKILL.md was NOT updated (gate did not pass)") if args.output: args.output.write_text(result.skill_after, encoding="utf-8") print(f"[scheduler] Output written to: {args.output}") ``` The declared entry point runs the automatic cycle directly: ```python from skill_scheduler import run_from_skill_dir, format_cycle_report result = run_from_skill_dir(ROBOT_EVOLVE_DIR, max_edits=6) print(format_cycle_report(result)) ``` ### Technical Analysis `SKILL.md` explicitly requires showing a diff and waiting for affirmative user confirmation before overwriting a skill file. The implementation does not enforce that security boundary. Instead, it treats an automated LLM-based gate result as authorization and immediately writes the remotely generated instructions into `SKILL.md`. The proposed edits originate from an external model prompt containing trajectory-derived text. Trajectory fields and target skill content are not treated as untrusted prompt input. An attacker able to influence a rollout, failure reason, feedback, session-state content, or target skill text can att ...[truncated 2230 chars]
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Error
Location
scripts/state_manager.py:42
Finding

Unsanitized Skill Identifier Enables Path Traversal During State File Access

Content
View full analysis
Path: """返回 state/.json 路径""" return STATE_DIR / f"{skill_id}.json" def load_state(skill_id: str) -> Optional[EvolveState]: """加载指定技能的状态,如果不存在则返回 None""" path = get_state_path(skill_id) if not path.exists(): return None data = json.loads(path.read_text(encoding="utf-8")) return EvolveState(**data) def save_state(state: EvolveState) -> None: """保存状态到 state/.json""" STATE_DIR.mkdir(exist_ok=True) state.last_update = datetime.now().isoformat() path = get_state_path(state.skill_id) path.write_text( json.dumps(asdict(state), ensure_ascii=False, indent=2), encoding="utf-8" ) ``` ### Technical Analysis `skill_id` is inserted directly into a filesystem path without validation or containment checking. Values containing `../` components can escape the intended `state` directory. An absolute path can also cause `pathlib` to discard the `STATE_DIR` prefix. The scheduler obtains `skill_id` from command-line input or from a target skill's `skill.json`. Therefore, a malicious or untrusted target skill package can supply a crafted name that redirects state reads and writes. The `.json` suffix limits the final filename but does not prevent traversal or overwriting arbitrary JSON files. The process writes a serialized `EvolveState`, so the attacker cannot write unrestricted byte content through this function, but can overwrite accessible JSON files with attacker-influenced state values. ### Attack Path 1. An attacker supplies a target skill whose `skill.json` contains a name such as `../../outside/target`, or invokes the scheduler with a traversal-containing skill identifier. 2. The scheduler passes that identifier to `load ...[truncated 1044 chars]
Remediation
View remediation
Path: if not _SKILL_ID_RE.fullmatch(skill_id): raise ValueError("Invalid skill identifier") path = (STATE_DIR / f"{skill_id}.json").resolve() state_root = STATE_DIR.resolve() if not path.is_relative_to(state_root): raise ValueError("State path escapes the state directory") return path ``` 2. Reject absolute paths, path separators, `.` segments, and `..` segments. 3. Do not trust the `name` field of a target skill package as a filesystem component. 4. Use atomic creation and replacement with restrictive file permissions. 5. Avoid following symbolic links when writing state files where platform support allows it. 6. Add traversal tests covering relative paths, absolute paths, mixed separators, encoded separators, and symlink escapes. ]]>
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Taint TrackingDirect Taint Flow, Variable-Mediated Taint Flow, Credential Exfiltration Chain
Findings (47)

Tainted flow: 'req' from os.environ.get (line 161, credential/environment) → urllib.request.urlopen (network output)

Critical
Category
Data Flow
Confidence
95% confidence
Finding

The request URL is derived from environment/config input (DEEPSEEK_BASE_URL) and the function sends sensitive skill content and failed trajectories to that endpoint without validation. If an attacker can influence the environment or config, they can redirect the data to an arbitrary server, causing exfiltration of potentially sensitive prompts, documents, and operational metadata.

Content

Scanner excerpt · scripts/evolve_core.py (reported line 170)May include surrounding context.

python
},
                method="POST",
            )
            with urllib.request.urlopen(req, timeout=120) as resp:
                result = json.loads(resp.read().decode("utf-8"))
            return result["choices"][0]["message"]["content"]
        except urllib.error.HTTPError as e:

Hidden Instructions

High
Category
Prompt Injection
Confidence
60% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · README.md (reported line 1)May include surrounding context.

md
# skill-evolve-pro — 技能进化引擎(Phase 1 + Phase 2)

> **版本**:Phase 1 + Phase 2  
> **基于**:SkillOpt / ReflACT 6步循环

Hidden Instructions

High
Category
Prompt Injection
Confidence
60% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 1)May include surrounding context.

md
# skill-evolve-pro — 技能进化引擎(Phase 1 + Phase 2)

> **版本**:Phase 1 + Phase 2  
> **基于**:SkillOpt / ReflACT 6步循环

Missing User Warnings

High
Category
Not specified by scanner
Confidence
99% confidence
Finding

The skill explicitly sends failure trajectories to the external DeepSeek API for analysis, but does not clearly warn that user data and model outputs may be transmitted to a third-party service. This is especially dangerous here because the trajectories can include raw user messages, predictions, references, and failure reasons, which may contain confidential or personal information.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

High
Category
Not specified by scanner
Confidence
99% confidence
Finding

The file contains a hardcoded DeepSeek API key as the default value for DEEPSEEK_API_KEY. Embedding a live secret in source code risks accidental disclosure through source control, package distribution, logs, or downstream reuse, and the surrounding docstring claiming the config is not uploaded can create false confidence that reduces safe handling of the secret.

Content

No source excerpt is available for this finding.

Self-Modification

High
Category
Rogue Agent
Confidence
92% confidence
Finding

This scheduler is explicitly designed to self-modify a skill by generating edits from rollout data, applying them, and writing the updated SKILL.md back to disk when the gate passes. Because the edit source is derived from potentially untrusted trajectories and reflection logic, a weak or bypassable gate can allow prompt/content injection, persistence of unsafe instructions, or corruption of the skill's behavior. In this context, the self-modification capability makes the system materially more dangerous because it persists generated changes rather than keeping them ephemeral.

Content

Scanner excerpt · scripts/skill_scheduler.py (reported line 11)May include surrounding context.

python
2. skill_reflect.reflect() -> generate edits from trajectories
  3. skill_apply.apply_all_edits() -> apply edits, produce skill_after
  4. skill_gate.gate_validate() -> validate (auto or manual)
  5. If gate passes -> write skill file -> bump_round -> record_edits -> save_state
"""

from __future__ import annotations

Ssd 4

High
Category
Not specified by scanner
Confidence
96% confidence
Finding

Model-generated guidance is automatically written into a protected long-term region without human review or robust validation. Because that region is intended to persist across updates, an injected or manipulated model output can become durable control-plane instruction, effectively escalating influence over future agent behavior.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The README states that the tool will automatically apply edits and produce an evolved skill version, but it does not prominently warn users that local skill files will be modified. In this context, silent or poorly disclosed file mutation is dangerous because the skill is specifically designed to rewrite code or prompt files, including protected-region updates in later phases.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The trigger phrases are very broad generic utterances such as '优化这个技能' and '让技能自动进化', which can overlap with ordinary user requests and cause the skill to activate unintentionally. Because this skill performs autonomous editing of other skills, accidental invocation can lead to unauthorized or surprising file modifications in the workspace.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
84% confidence
Finding

文档标题、流程说明、用户确认词和示例几乎全部固定为中文,例如要求回复“确认改”“同意”“确认”等,但没有说明是否支持其他语言或允许用户自行选择交互语言。按规则,这属于可能的语言/locale 强制约束,且缺少用户 opt-in。

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The trigger phrases include very common requests such as '优化这个技能' and '运行进化流程', which can plausibly appear in normal discussion and cause unintended activation of a workflow that edits files. In this skill's context, accidental triggering is more dangerous because the skill is designed to modify other skills and process stored trajectory data, so a false activation could lead to unwanted analysis, diff generation, or preparation for file changes.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

Phase 2 describes automatic parsing of SESSION-STATE.md, including user messages, feedback, and task metadata, but does not state any notice, minimization, redaction, retention, or consent controls. In context, this is risky because session-state content may contain sensitive user data and is being repurposed for rollout analysis beyond the immediate interaction.

Content

No source excerpt is available for this finding.

Ssd 3

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The skill instructs automatic collection, storage, and later reuse of raw user messages, feedback, and related rollout metadata from SESSION-STATE. This creates a data-retention and secondary-use risk because sensitive interaction content may be persisted in temp/rollouts without clear minimization, retention, or access-control guidance.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The module docstring is written entirely in Chinese and includes an instruction-like description without offering any language choice or documenting a justified locale restriction. Under the policy, forcing a specific language or locale without user opt-in is a natural-language policy violation.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

This code file contains user-facing natural-language text entirely in Chinese, including the module description and setup instructions. The file does not offer a language choice or indicate that the locale restriction is intentional for a region-specific tool, which fits the policy category for forced language without user opt-in.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The RuntimeError text shown to users when configuration is missing is Chinese-only. Because this is a direct user-facing interaction in code and there is no alternative language path or documented justification, it constitutes a natural-language policy violation under the locale rule.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
86% confidence
Finding

The file’s natural-language strings and system prompts are written exclusively in Chinese, which effectively constrains operation to a specific language/locale without any opt-in or alternative. The policy requires flagging language or locale constraints when the skill forces a specific language without giving the user a choice.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
85% confidence
Finding

The Reflect system prompt and surrounding documentation repeatedly instruct the model to generate changes for "SKILL.md", implying the tool is scoped to skill-document evolution. However, the CLI at L649-L654 reads an arbitrary path from argv and L619 applies edits to whatever content was supplied, so the documented target is narrower than the implemented behavior.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The evolve cycle transmits portions of the skill document and failed trajectory data to an external LLM service, but this code provides no user-facing warning, consent gate, or redaction step. In security-sensitive environments, that can leak proprietary, personal, or confidential data to a third party, especially because the transfer is integral to normal operation and easy to overlook.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
87% confidence
Finding

This code repeatedly assumes the user is referred to as “先生” and uses that label in comments, parsing rules, and generated output. That imposes a specific language/locale-style form of address rather than offering a user choice, which fits the policy category for forced language/locale behavior.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The parser only recognizes user lines prefixed with “先生” and AI lines prefixed with “AI” or “双鱼座”, which enforces a particular localized naming convention. Users who do not use that form are not given any documented choice or configuration, making this a natural-language policy concern.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
70% confidence
Finding

npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
70% confidence
Finding

npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
70% confidence
Finding

npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
70% confidence
Finding

npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.

Content

No source excerpt is available for this finding.