Back to skill

Security audit

skill-creator

Security checks for vulnerabilities and agentic risk

Overview

The skill is a coherent skill-building and evaluation tool, but its review helpers have unsafe local side effects and browser-injection risks that users should review before installing.

Install only if you are comfortable running a local skill-development tool that writes evaluation workspaces, invokes the Claude CLI, opens local/browser review pages, and stores feedback files. Avoid running it with elevated privileges. Prefer the static viewer or an unused random port, and be cautious evaluating untrusted skills until the HTML/JavaScript embedding is fixed with context-safe escaping or non-executable JSON storage. After use, confirm any background viewer is stopped and temporary .claude command files were removed.

Vulnerability Patterns
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (3)

T09 · Insecure Skill Coding Practices

Error
Location
eval-viewer/generate_review.py:279
Finding

Stored Script Injection Through Embedded Evaluation Data

Content
View full analysis
// ---- Embedded data (injected by generate_review.py) ---- /*__EMBEDDED_DATA__*/ ``` ### Technical Analysis `generate_html()` serializes evaluation data with `json.dumps()` and inserts the result directly into an executable HTML ``. An evaluated output containing a payload such as: ```html ``` would terminate the original script element before JavaScript parsing and introduce a new executable script element. Although most viewer rendering uses `textContent`, that protection occurs only after `EMBEDDED_DATA` has been parsed. It does not protect the initial server-generated script block. ### Attack Path 1. An untrusted or compromised Skill is included in an evaluation. 2. The Skill generates a text output containing a `` sequence followed by attacker-controlled JavaScript. 3. `embed_file()` reads the generated output and includes it in the `embedded` data structure. 4. `generate_html()` serializes the structure and places it directly inside the viewer’s executable script block. 5. The user opens or refreshes the evaluation viewer. 6. The browser terminates the intended script block and executes the injected JavaScript in th ...[truncated 998 chars]
Remediation
View remediation
``` Then retrieve and parse its text: ```javascript const EMBEDDED_DATA = JSON.parse( document.getElementById("embedded-data").textContent ); ``` 3. Before embedding JSON into HTML, escape characters significant to the HTML parser. At minimum: ```python data_json = ( json.dumps(embedded) .replace("&", "\\u0026") .replace("<", "\\u003c") .replace(">", "\\u003e") .replace("\u2028", "\\u2028") .replace("\u2029", "\\u2029") ) ``` Escaping `<` prevents creation of a literal `` sequence. 4. Add regression tests using output values containing: ```text

T09 · Insecure Skill Coding Practices

Error
Location
assets/eval_review.html:6
Finding

Script and HTML Injection in the Eval-Set Review Template

Content
View full analysis
Eval Set Review - __SKILL_NAME_PLACEHOLDER__ ``` ```html

Eval Set Review: __SKILL_NAME_PLACEHOLDER__

Current description: __SKILL_DESCRIPTION_PLACEHOLDER__

``` It also places evaluation data directly into an executable script context: ```html ` even if the array is serialized as otherwise valid JSON. Escaping query text later with `escapeHtml()` does not mitigate this issue beca ...[truncated 1516 chars]
Remediation
View remediation
``` ```javascript document.getElementById("skill-name").textContent = metadata.skillName; document.getElementById("skill-desc").textContent = metadata.skillDescription; ``` 3. Store eval data in a non-executable JSON element: ```html ``` Parse it with: ```javascript const EVAL_DATA = JSON.parse( document.getElementById("eval-data").textContent ); ``` 4. Escape `<`, `>`, `&`, U+2028, and U+2029 in serialized JSON. In particular, encode `<` as `\u003c` so `` cannot appear literally in the HTML source. 5. HTML-escape any data inserted into `` or normal element-body contexts. 6. Avoid deriving temporary filenames directly from an unvalidated Skill name. Use a generated safe filename or strictly normalize it. 7. Add a restrictive Content Security Policy and avoid executable inline scripts where practical. 8. Add automated tests covering malicious Skill metadata and queries containing closing tags, quotes, event handlers, and script terminators. ]]>

T05 · Unauthorized Access and Privilege Escalation

Warning
Location
eval-viewer/generate_review.py:287
Finding

Evaluation Viewer Terminates Unrelated Processes on Its Selected Port

Content
View full analysis
None: """Kill any process listening on the given port.""" try: result = subprocess.run( ["lsof", "-ti", f":{port}"], capture_output=True, text=True, timeout=5, ) for pid_str in result.stdout.strip().split("\n"): if pid_str.strip(): try: os.kill(int(pid_str.strip()), signal.SIGTERM) except (ProcessLookupError, ValueError): pass if result.stdout.strip(): time.sleep(0.5) except subprocess.TimeoutExpired: pass except FileNotFoundError: print("Note: lsof not found, cannot check if port is in use", file=sys.stderr) ``` The function is called unconditionally before attempting to bind the server: ```python # Kill any existing process on the target port port = args.port _kill_port(port) ``` The code subsequently demonstrates that safe fallback behavior is already available: ```python try: server = HTTPServer(("127.0.0.1", port), handler) except OSError: # Port still in use after kill attempt — find a free one server = HTTPServer(("127.0.0.1", 0), handler) port = server.server_address[1] ``` ### Technical Analysis The viewer needs a local listening port, but it does not need control over a specific fixed port. `_kill_port()` obtains every PID reported by `lsof` for the selected port and sends `SIGTERM` without checking: - Whether the process belongs to this viewer. - Whether the process belongs to the current user. - Whether the process is critical or unrelated. - Whether the user consented to termination. - Whether selecting another port would satisfy the task. The use of an argument array prevents shel ...[truncated 1302 chars]
Remediation
View remediation
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Rogue AgentSelf-Modification, Session Persistence
Findings (20)

Hidden Instructions

High
Category
Prompt Injection
Confidence
60% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 1)May include surrounding context.

md
---
name: skill-creator
description: "Create, modify, optimize skills with eval testing, benchmark analysis, and description optimization"
tags: [coding, data, visual, file-based, iterative]

Natural-Language Policy Violations

High
Category
Not specified by scanner
Confidence
99% confidence
Finding

The file includes a substantial Chinese-language section without any gating, translation, or indication that multilingual content is expected. This can create instruction opacity for users and models operating in English, increasing the chance that important behavioral requirements are missed or misapplied and reducing reviewability of the skill’s logic.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
96% confidence
Finding

The skill explicitly instructs authors to make descriptions "pushy" to counter undertriggering, which encourages overbroad matching and increases the chance the skill activates for unrelated requests. In a skill system where descriptions govern invocation, this can cause unintended execution of complex workflows, extra tool usage, and user-surprising behavior beyond the user’s actual intent.

Content

No source excerpt is available for this finding.

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 415)May include surrounding context.

md
1. Read the template from `assets/eval_review.html`

Env Variable Harvesting

High
Category
Data Exfiltration
Confidence
70% confidence
Finding

Code enumerates, copies, or searches environment variables for secrets. Bulk environment access can collect credentials unrelated to the skill's stated purpose.

Content

Scanner excerpt · scripts/improve_description.py (reported line 33)May include surrounding context.

python
# Remove CLAUDECODE env var to allow nesting claude -p inside a
    # Claude Code session. The guard is for interactive terminal conflicts;
    # programmatic subprocess usage is safe. Same pattern as run_eval.py.
    env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

    result = subprocess.run(
        cmd,

Env Variable Harvesting

High
Category
Data Exfiltration
Confidence
70% confidence
Finding

Code enumerates, copies, or searches environment variables for secrets. Bulk environment access can collect credentials unrelated to the skill's stated purpose.

Content

Scanner excerpt · scripts/run_eval.py (reported line 83)May include surrounding context.

python
# Remove CLAUDECODE env var to allow nesting claude -p inside a
    # Claude Code session. The guard is for interactive terminal conflicts;
    # programmatic subprocess usage is safe. Same pattern as run_eval.py.
    env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

    result = subprocess.run(
        cmd,

Agent Config Directory Access

High
Category
Agent Snooping
Confidence
89% confidence
Finding

The script intentionally discovers the project root by locating a .claude/ directory and then writes a temporary command file into .claude/commands. Accessing and modifying an agent configuration directory can influence agent behavior and, if run in an unexpected repository or shared workspace, may alter trusted local agent state or interfere with other commands.

Content

Scanner excerpt · scripts/run_eval.py (reported line 23)May include surrounding context.

python
def find_project_root() -> Path:
    """Find the project root by walking up from cwd looking for .claude/.

    Mimics how Claude Code discovers its project root, so the command file
    we create ends up where claude -p will look for it.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
87% confidence
Finding

The skill says to "figure out where the user is in this process and then jump in and help them progress through these stages," but gives weak limits on when not to activate. That ambiguity broadens the operational scope of the skill and can lead to accidental invocation for general brainstorming, coding help, or evaluation tasks that do not actually require this skill.

Content

No source excerpt is available for this finding.

Session Persistence

Medium
Category
Rogue Agent
Confidence
78% confidence
Finding

The skill instructs launching a background process with nohup, which persists beyond the immediate interaction and can continue consuming resources or exposing locally served review artifacts if not properly terminated. Because the workflow also mentions capturing a PID and later killing it, the persistence is partially controlled, but it still increases operational risk if cleanup is skipped or fails.

Content

Scanner excerpt · SKILL.md (reported line 289)May include surrounding context.

  1. Launch the viewer with both qualitative outputs and quantitative data:
    bash
    nohup python <skill-creator-path>/eval-viewer/generate_review.py \
      <workspace>/iteration-N \
      --skill-name "my-skill" \
      --benchmark <workspace>/iteration-N/benchmark.json \
    

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The file first defines the 'Post-hoc Analyzer Agent' as analyzing a winner/loser comparison and generating actionable improvement suggestions for the losing skill, including a structured JSON output with 'improvement_suggestions'. Later, the 'Analyzing Benchmark Results' section states that the analyzer's purpose is instead to surface benchmark patterns and explicitly says 'DO NOT suggest improvements to the skill,' which is a direct contradiction in intent within the same skill documentation.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The script kills whatever process is listening on the requested port before starting its own server, even though serving a review page does not require destructive system actions. This can terminate unrelated local services, developer tools, or security-sensitive processes, causing denial of service and creating an unsafe side effect disproportionate to the tool's stated purpose.

Content

No source excerpt is available for this finding.

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
90% confidence
Finding

The subprocess call itself is not shell-injection prone because it uses an argument list and the port is parsed as an integer, but it is part of functionality that enumerates and enables termination of arbitrary local processes. In this review tool context, invoking an external system utility to identify PIDs on a port creates unnecessary administrative reach and increases the blast radius of running the script.

Content

Scanner excerpt · eval-viewer/generate_review.py (reported line 291)May include surrounding context.

python
def _kill_port(port: int) -> None:
    """Kill any process listening on the given port."""
    try:
        result = subprocess.run(
            ["lsof", "-ti", f":{port}"],
            capture_output=True, text=True, timeout=5,
        )

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

User-entered feedback is auto-saved to /api/feedback after typing, but the UI does not clearly disclose that content is being transmitted to a server before explicit submission. In a review tool, that can leak draft comments, sensitive evaluation notes, or proprietary data to a backend unexpectedly, especially if users assume text remains local until they click submit.

Content

No source excerpt is available for this finding.

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · scripts/improve_description.py (reported line 35)May include surrounding context.

python
# programmatic subprocess usage is safe. Same pattern as run_eval.py.
    env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

    result = subprocess.run(
        cmd,
        input=prompt,
        capture_output=True,

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · scripts/run_eval.py (reported line 85)May include surrounding context.

python
# programmatic subprocess usage is safe.
        env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

        process = subprocess.Popen(
            cmd,
            stdout=subprocess.PIPE,
            stderr=subprocess.DEVNULL,

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
78% confidence
Finding

This markdown skill instructs the agent to save analysis results to {output_path}, which is a file-writing operation. The document does not include any warning, confirmation, or disclosure that it will write to the provided path or overwrite existing contents.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
85% confidence
Finding

This markdown skill instructs the agent to save results to a sibling grading.json file, which is a file write that affects user data/workspace state. The description does not include any warning or disclosure that the skill will create or overwrite this file.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Low
Category
Not specified by scanner
Confidence
93% confidence
Finding

The top-level docstring explicitly claims no dependencies beyond the Python standard library. But _kill_port executes lsof, which is not part of the Python stdlib and may not be installed, so the documentation materially misstates what the script depends on for its advertised behavior.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
87% confidence
Finding

The code retrieves prior reviews from /api/feedback, which is a network operation involving user review data. There is no clear warning or disclosure in the visible page text that previously saved feedback will be loaded from a server.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Low
Category
Not specified by scanner
Confidence
92% confidence
Finding

The comment at L1027 says the current textarea is saved to feedbackMap "but don't POST yet," implying no network submission at that stage. However, the same function proceeds to build a payload and POST it to /api/feedback at L1044-L1048 as part of the submit flow. This is a direct contradiction between inline documentation and actual behavior.

Content

No source excerpt is available for this finding.