Security audit

skills creator

Security checks for vulnerabilities and agentic risk

Overview

This skill is a coherent skill-generation framework, but it can run generated code locally and write new skill files in ways that are too powerful and under-scoped for normal installation.

Install only in an isolated development environment. Do not use it to test untrusted generated skills unless you replace the local runners with a real sandbox and disable the Docker-to-local fallback. Review generated SKILL.md and Python files before making them loadable, constrain skill names to safe identifiers, and be aware that LLM/API use and model downloads may send or fetch data from external providers.

A.I.G

Vulnerability Patterns

Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
Insecure DependenciesIntroduces malicious components through unsafe dependency sources
Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review

Findings (6)

T01 · Skill Instruction Hijacking

Error

Location: generated/truthfulness_optimized.md:11
Finding: Generated Skill Attempts to Override the Agent Instruction Hierarchy
Content: View full analysis
Remediation: View remediation

T09 · Insecure Skill Coding Practices

Error

Location: scripts/sandbox.py:27
Finding: SimpleSandbox Executes Untrusted Python Directly on the Host
Content: View full analysis
Remediation: View remediation

T09 · Insecure Skill Coding Practices

Error

Location: scripts/tester.py:58
Finding: Docker Tester Silently Falls Back to In-Process Arbitrary Code Execution
Content: View full analysis
', 'exec') # 执行代码 exec(compiled, namespace) ``` ```python def _get_client(self): """获取 Docker 客户端""" try: import docker if not self.client: self.client = docker.from_env() return self.client except ImportError: print("⚠️ Docker SDK 未安装，回退到简单测试器") return None except Exception as e: print(f"⚠️ Docker 连接失败: {e}，回退到简单测试器") return None def test_skill(self, code: str, test_cases: List[TestCase]) -> List[TestResult]: """在 Docker 容器中测试""" client = self._get_client() if not client: # 回退到简单测试器 tester = SimpleTester(self.timeout) return tester.test_skill(code, test_cases) ``` ### Technical Analysis `SimpleTester` compiles and executes supplied Python inside the framework process with unrestricted built-ins. The namespace does not provide meaningful isolation because code can import modules, access the filesystem and network, inspect process state, and terminate or alter the host process. `DockerTester` automatically selects this unsafe execution path whenever the Docker SDK is absent or the Docker daemon cannot be reached. A security control failure therefore causes execution to become less secure rather than stopping safely. The thread-based timeout used by `SimpleTester` is also insufficient. Python cannot safely terminate the timed-out worker thread, so malicious code may continue executing after the framework reports a timeout. ### Attack Path 1. An attacker supplies or induces generation of malicious Python code. 2. The caller selects `DockerTester`, believing the code wi ...[truncated 716 chars]
Remediation: View remediation

T09 · Insecure Skill Coding Practices

Error

Location: scripts/generate_skill.py:21
Finding: Unsanitized Skill Names Permit Arbitrary File Writes Through Path Traversal
Content: View full analysis
dict: """Generate a new skill based on description""" # Convert name to hyphen-case skill_id = skill_name.lower().replace(" ", "-") ``` ```python def save_generated_skill(skill_data: dict): """Save generated skill to disk""" GENERATED_DIR.mkdir(parents=True, exist_ok=True) skill_dir = GENERATED_DIR / skill_data["name"] skill_dir.mkdir(parents=True, exist_ok=True) # Write SKILL.md skill_file = skill_dir / "SKILL.md" skill_file.write_text(skill_data["content"], encoding="utf-8") return str(skill_dir) ``` ### Technical Analysis The only transformation applied to `skill_name` is lowercasing and replacing spaces with hyphens. Directory separators, absolute paths, drive-qualified paths, and `..` traversal segments are not rejected. When a traversal-bearing value is passed to `Path`, the resulting destination can escape `GENERATED_DIR`. If the supplied name is absolute, normal path-joining behavior can discard the intended base path entirely. The final write uses `write_text()`, which overwrites an existing `SKILL.md` if the process has permission. ### Attack Path 1. An attacker supplies a crafted Skill name such as `../../target` or an absolute writable path. 2. `generate_skill()` preserves the path separators and traversal components. 3. `save_generated_skill()` combines the attacker-controlled name with `GENERATED_DIR`. 4. `mkdir()` creates directories outside the intended generated-Skill area. 5. `write_text()` creates or overwrites `SKILL.md` at the attacker-selected writable location. ### Impact Assessment The attacker can create directories and overwrite files named `SKILL.md` anywhere wri ...[truncated 250 chars]
Remediation: View remediation

T09 · Insecure Skill Coding Practices

Error

Location: scripts/auto_refactor.py:89
Finding: Automatic Refactoring Uses Untrusted Skill Identifiers as Output Paths
Content: View full analysis
Optional[RefactorSuggestion]: """生成 meta-skill""" skill_name = "meta_{}_{}".format(skill_a, skill_b) ``` ```python # 保存 output_file = os.path.join(self.output_dir, "{}.py".format(skill_name)) with open(output_file, 'w', encoding='utf-8') as f: f.write(code) ``` ### Technical Analysis Skill identifiers are recorded in persistent usage history without validation. Once a pair reaches the configured frequency threshold, the identifiers are interpolated directly into a Python filename. `os.path.join()` does not establish a containment boundary. Traversal components in `skill_a` or `skill_b` can cause the resolved filename to escape `output_dir`. The use of write mode also permits overwriting an existing writable file. This is a stored exploitation path: malicious identifiers can remain in the history file until automatic refactoring processes them. ### Attack Path 1. An attacker causes traversal-bearing Skill identifiers to be included in execution sequences. 2. `record_execution()` stores those identifiers in `logs/skill_usage.json`. 3. The malicious pair is recorded enough times to satisfy `min_usage_count`. 4. `run_auto_refactor()` selects the pair as frequent. 5. `generate_meta_skill()` embeds the identifiers in `skill_name`. 6. `os.path.join()` produces a path outside `output_dir`. 7. The generated Python source is written to or over an attacker-selected writable `.py` path. ### Impact Assessmen ...[truncated 303 chars]
Remediation: View remediation

T08 · Insecure Dependencies

Warning

Location: scripts/vector_search.py:9
Finding: Mutable Model Artifacts Are Downloaded Through an Unpinned Third-Party Mirror
Content: View full analysis
Remediation: View remediation

SkillSpector

Vulnerability Patterns

Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Rogue AgentSelf-Modification, Session Persistence
Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
Behavioral ASTexec() Call, eval() Call, Dynamic Import
MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration

Findings (70)

Tp4

High

Category: MCP Tool Poisoning
Confidence: 97% confidence
Finding: 声明描述的是一个完整的“技能工厂/自动生成框架”核心功能，范围涵盖扫描、注册、检索、生成、评估、测试、优化以及向量搜索库相关流程。但提供的代码片段仅是一个基础抽象类模板，用于约束生成技能的接口与错误处理，属于底层支撑组件，而不是所声明的核心框架功能实现。虽然该模板可能是框架的一部分，但就这段代码本身而言，其实际行为与声明的主要目的存在明显差异，因此应判定为不匹配。
Content: No source excerpt is available for this finding.

Tp4

High

Category: MCP Tool Poisoning
Confidence: 95% confidence
Finding: 声明描述的是一个较完整的“技能工厂/自动生成框架”，核心包括扫描、注册、检索、生成、评估、测试、优化，以及向量搜索库相关操作。但给定代码片段只覆盖其中很窄的一部分：它把技能调用序列写入本地 JSON 日志，统计高频技能组合，并用模板生成组合技能文件到本地目录。其主要目的更接近“基于使用历史的自动重构器/组合技能生成器”，而不是通用的技能扫描、注册、检索和评估测试框架。此外，描述中特别提到首次使用需运行 embed_skill.py 重建向量搜索库，但该代码完全未涉及向量库、embedding、索引或检索。因此，代码行为与声明用途存在明显且实质性的不一致。
Content: No source excerpt is available for this finding.

Tp4

High

Category: MCP Tool Poisoning
Confidence: 96% confidence
Finding: 描述宣称的是一个较完整的“技能自动生成框架”，包括扫描、注册、检索、生成、评估、测试、优化等多项核心功能，且状态标注为“核心功能已完成”。但提供的代码块仅是 scripts/embed_skill.py，一个向量数据库写入与查询工具。它会读取 metadata.yaml，将技能描述与标签写入 ChromaDB，并提供查询、列举、删除、重复检测等辅助管理功能。虽然这与声明中的“注册/检索”部分一致，也与“首次使用需运行 python scripts/embed_skill.py 重建向量搜索库”的提示相符，但它并未体现“生成、评估、测试、优化技能”的能力，主功能范围明显更窄。因此，描述对该代码块的能力存在实质性夸大，属于不准确表述。
Content: No source excerpt is available for this finding.

Tp4

High

Category: MCP Tool Poisoning
Confidence: 90% confidence
Finding: 代码的核心行为是本地生成技能元数据与 Markdown 文件并写入 generated 目录，属于“生成技能”的单一子功能。声明则描述了一个更完整的技能工厂/框架，覆盖扫描、注册、检索、评估、测试、优化等多项能力，并特别提到 embedding/向量搜索库初始化。当前片段没有展示这些能力的实现，也没有访问或构建向量搜索库；虽然定义了 load_db() 读取 skills_db.json，但在该片段中未实际用于注册或检索。因此，声明的范围明显大于代码实际行为，构成描述与行为不一致。
Content: No source excerpt is available for this finding.

Tp4

High

Category: MCP Tool Poisoning
Confidence: 92% confidence
Finding: 该代码块的核心行为是“技能生成器”，重点在于根据需求用 LLM 或模板产出 Python 技能类，并提供一个技能代码模板。这与声明中的‘自动生成框架’只部分重合。声明强调的是一个覆盖扫描、注册、检索、生成、评估、测试、优化的完整框架，且特别提到向量搜索库重建；但当前代码中看不到任何扫描目录、注册技能、检索技能、嵌入/向量库、评估、测试执行、优化改写等实现。因此描述明显宽于实际行为，属于功能性不匹配。
Content: No source excerpt is available for this finding.

Tp4

High

Category: MCP Tool Poisoning
Confidence: 87% confidence
Finding: The declared description presents a broader 'AI skill auto-generation framework' with capabilities for scanning, registration, retrieval, generation, evaluation, testing, and optimization, and specifically references rebuilding a vector search library with an embedding script. The supplied code chunk does not implement most of those capabilities. It only manages a JSON file of skills and performs simple keyword-based search without embeddings. While registration and retrieval are partially consistent with the description, the primary behavior here is a lightweight registry/search helper rather than the full framework described. Therefore this is a meaningful description-to-behavior mismatch.
Content: No source excerpt is available for this finding.

Tp4

High

Category: MCP Tool Poisoning
Confidence: 97% confidence
Finding: 该代码片段与声明的“AI 技能自动生成框架”核心能力明显不一致。描述声称系统具备扫描、注册、检索、生成、评估、测试、优化技能等完整框架能力，但这段代码只做基础验证：检查文件是否存在、读取文件内容、编译做语法检查、验证配置和模板字段。虽然声明中包含“测试技能代码”，而该脚本确实属于测试相关支持功能，但它并未体现描述中的主要或综合能力，尤其没有任何自动生成、向量检索、注册、评估、优化等实现。因此应判定为描述与实际行为存在明显不匹配。
Content: No source excerpt is available for this finding.

Self-Modification

High

Category: Rogue Agent
Confidence: 87% confidence
Finding: The script writes AI-generated content directly to the repository under the skills tree, enabling persistent self-modification of the agent's skill set. In this meta-skill context, untrusted input in skill_name, description, or trigger_phrases can become a new executable/trusted skill artifact, which may later be loaded and influence agent behavior, creating a persistence and indirect prompt/code injection risk.
Content: Scanner excerpt · scripts/generate_skill.py (reported line 84)May include surrounding context.
python
skill_dir = GENERATED_DIR / skill_data["name"] skill_dir.mkdir(parents=True, exist_ok=True) # Write SKILL.md skill_file = skill_dir / "SKILL.md" skill_file.write_text(skill_data["content"], encoding="utf-8")

Intent-Code Divergence

High

Category: Not specified by scanner
Confidence: 99% confidence
Finding: The class is presented as a sandbox, but it provides no sandboxing boundary beyond launching a subprocess. That mismatch is dangerous because developers may trust it to safely test generated or retrieved skills, when in reality any supplied code runs with host privileges.
Content: No source excerpt is available for this finding.

Missing User Warnings

High

Category: Not specified by scanner
Confidence: 95% confidence
Finding: The skill executes untrusted code as part of a testing workflow without an explicit warning, consent gate, or trust boundary, which increases the chance that operators run hostile input assuming it is safely contained. In this skill's context—automatic generation, evaluation, and testing of skills—that risk is elevated because the system is designed to handle third-party or model-generated code at scale.
Content: No source excerpt is available for this finding.

Intent-Code Divergence

High

Category: Not specified by scanner
Confidence: 98% confidence
Finding: The module advertises a 'secure sandbox execution' capability, but SimpleTester executes arbitrary code locally without real isolation. This mismatch is dangerous because users may trust the component and run hostile skills under false assumptions, increasing the likelihood of host compromise.
Content: No source excerpt is available for this finding.

Missing User Warnings

High

Category: Not specified by scanner
Confidence: 96% confidence
Finding: The code path performs local execution of untrusted skill code without any explicit warning, confirmation, or unsafe-mode flag. In a skill-generation/testing framework, this significantly raises risk because users may assume submitted skills are merely being validated rather than executed with host privileges.
Content: No source excerpt is available for this finding.

exec() call detected

High

Category: Dangerous Code Execution
Confidence: 100% confidence
Finding: exec(compiled, namespace) runs untrusted skill code with full Python builtins in the local process, so a malicious skill can import os/subprocess, read secrets, modify files, or spawn processes. The provided 'isolated namespace' is not a security boundary, and the thread-based timeout does not prevent harmful side effects before timeout triggers.
Content: Scanner excerpt · scripts/tester.py (reported line 69)May include surrounding context.
python
compiled = compile(code, '<skill>', 'exec') # 执行代码 exec(compiled, namespace) # 查找技能类 skill_class = None

Docker Socket Access

High

Category: Privilege Escalation
Confidence: 85% confidence
Finding: docker.from_env() commonly connects via the local Docker socket, and access to that socket is effectively root-equivalent on the host because it can start privileged containers, mount host paths, and access secrets. In a testing component for untrusted skills, exposing or relying on broad Docker daemon access increases the blast radius if surrounding code, configuration, or credentials are abused.
Content: Scanner excerpt · scripts/tester.py (reported line 188)May include surrounding context.
python
try: import docker if not self.client: self.client = docker.from_env() return self.client except ImportError: print("⚠️ Docker SDK 未安装，回退到简单测试器")

Missing User Warnings

High

Category: Not specified by scanner
Confidence: 99% confidence
Finding: When Docker is unavailable, the system silently falls back from containerized testing to local in-process execution, converting a relatively safer path into arbitrary host code execution without informed consent. In this skill context, where the framework scans and generates third-party skills, that fallback materially increases the chance of executing hostile code from semi-automated workflows.
Content: No source excerpt is available for this finding.

Undeclared Tool Scope

Medium

Category: MCP Least Privilege
Confidence: 90% confidence
Finding: The skill advertises or implies capabilities involving shell, environment access, and file read/write, but it does not declare any explicit tool scope such as permissions or allowed-tools. In an agent system, missing scope boundaries can lead to over-privileged execution, making unintended file modification, command execution, or environment disclosure more likely if the skill is invoked.
Content: No source excerpt is available for this finding.

Vague Triggers

Medium

Category: Not specified by scanner
Confidence: 95% confidence
Finding: The trigger conditions are very broad, including generic requests to create a 'skill factory' or to retrieve, generate, evaluate, test, or optimize skills. In agent environments, ambiguous activation criteria can cause unintended invocation of a powerful skill, increasing the chance of unnecessary file access, code generation, or shell-related actions in contexts where the user did not clearly request them.
Content: No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium

Category: Not specified by scanner
Confidence: 95% confidence
Finding: This Python template contains user- and developer-facing natural language entirely in Chinese, including the module docstring, method docstrings, and the default error suggestion. As a reusable base skill template, this effectively imposes a specific language on downstream skills without any visible opt-in or locale-selection mechanism, matching the policy category for forced language/locale.
Content: No source excerpt is available for this finding.

Vague Triggers

Medium

Category: Not specified by scanner
Confidence: 87% confidence
Finding: Listing trigger phrases without boundaries, qualifiers, or negative examples makes activation ambiguous across languages and contexts. In an agent ecosystem, ambiguous triggers can lead to accidental invocation, misrouting, and unintended execution of reminder-related actions when the user did not mean to call this skill.
Content: No source excerpt is available for this finding.

Vague Triggers

Medium

Category: Not specified by scanner
Confidence: 95% confidence
Finding: The trigger phrase "alarm" is very broad and can appear in ordinary conversation unrelated to invoking this skill. This increases the chance of unintended activation or routing collisions with other skills, which can cause the wrong automation to run or capture user intent incorrectly.
Content: No source excerpt is available for this finding.

Vague Triggers

Medium

Category: Not specified by scanner
Confidence: 94% confidence
Finding: The activation conditions are very broad, including general cases like factual questions, providing numbers, uncertainty, or correcting prior errors. Because the skill also claims highest priority over all other instructions, it can trigger across a wide range of normal interactions and unexpectedly alter agent behavior, increasing the chance of instruction conflicts or unnecessary tool use.
Content: No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium

Category: Not specified by scanner
Confidence: 88% confidence
Finding: The manifest description and body of the skill are presented in Chinese, which can impose a language preference without user opt-in. The file does not indicate that the skill is region-specific or that users may select another language, so it may violate language/locale policy expectations.
Content: No source excerpt is available for this finding.

Vague Triggers

Medium

Category: Not specified by scanner
Confidence: 95% confidence
Finding: The embedded truthfulness skill declares triggers such as factual questions, use of numbers, prior errors, and uncertainty, then says it has the highest priority over all other instructions. Those conditions are broad enough to activate during many normal conversations and can unexpectedly override narrower safety, routing, or task-specific skills, creating prompt-scope capture and policy interference.
Content: No source excerpt is available for this finding.

Vague Triggers

Medium

Category: Not specified by scanner
Confidence: 95% confidence
Finding: The second truthfulness entry repeats the same expansive trigger conditions and global-priority wording, duplicating the same prompt-injection-by-design risk. Because it appears multiple times in an optimization database, the behavior may be reinforced or repeatedly surfaced by tooling that retrieves or reuses prior optimized skills, increasing the chance of unintended activation.
Content: No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium

Category: Not specified by scanner
Confidence: 96% confidence
Finding: This markdown file presents the skill documentation entirely in Chinese, while the visible headings and code context suggest no explicit user choice or opt-in for language. The policy category requires flagging language or locale constraints when the skill forces a specific language without offering alternatives or documenting a justified regional scope.
Content: No source excerpt is available for this finding.