Back to skill

Security audit

skill-creator

Security checks for vulnerabilities and agentic risk

Overview

The skill is mostly coherent for creating and testing skills, but its bundled review viewer can silently terminate unrelated local processes on a chosen port.

Install only if you are comfortable with a skill that edits local skill files, runs evaluation scripts and subagents, invokes the local claude CLI, starts localhost review servers, and writes review/benchmark artifacts. Run the viewer with --static when possible, or check/patch generate_review.py before use so it does not kill unrelated processes on port conflicts. Prefer project-scoped installation until the trigger behavior and local effects are acceptable.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Warning
Location
eval-viewer/generate_review.py:286
Finding

Viewer Startup Unconditionally Terminates Processes Using the Selected Port

Content
View full analysis

Vulnerability Details

File Location: eval-viewer/generate_review.py, lines 286–306 and 430–432
Vulnerability Type: Uncontrolled termination of an unrelated local process
Risk Level: Medium

Complete Code Snippet

python
def _kill_port(port: int) -> None:
    """Kill any process listening on the given port."""
    try:
        result = subprocess.run(
            ["lsof", "-ti", f":{port}"],
            capture_output=True, text=True, timeout=5,
        )
        for pid_str in result.stdout.strip().split("\n"):
            if pid_str.strip():
                try:
                    os.kill(int(pid_str.strip()), signal.SIGTERM)
                except (ProcessLookupError, ValueError):
                    pass
        if result.stdout.strip():
            time.sleep(0.5)
    except subprocess.TimeoutExpired:
        pass
    except FileNotFoundError:
        print("Note: lsof not found, cannot check if port is in use", file=sys.stderr)

The function is invoked unconditionally during viewer startup:

python
# Kill any existing process on the target port
port = args.port
_kill_port(port)

Technical Analysis

The evaluation viewer defaults to port 3117 but also accepts a user-selected port through --port. Before attempting to bind its HTTP server, it invokes lsof to identify every process listening on that port and sends each reported PID SIGTERM.

The code does not establish that the target process is a viewer previously created by this project. It does not verify the executable, command line, process owner, PID file, or viewer-specific instance token. It also does not require explicit confirmation or an opt-in replacement flag.

Consequently, authorization to launch the evaluation viewer is expanded into termination of an arbitrary same-user process outside the viewer's workspace and lifecycle. This is reachable through the documented workflow in SKILL.md, which instructs the agent to start `eval-viewer/generate_revie ...[truncated 1644 chars]

Remediation
View remediation

Remediation Suggestions

  1. Remove _kill_port() from the default startup path.
  2. Attempt to bind the requested port first. If it is unavailable, use the existing fallback that binds to port 0 and lets the operating system choose a free port.
  3. If replacing an existing viewer is required, make it explicitly opt-in, such as --replace-existing-viewer.
  4. Track viewer instances with a securely created PID file containing the PID and a viewer-specific identity token.
  5. Before terminating a tracked process, verify that:
    • the PID still refers to the expected process;
    • the process belongs to the current user;
    • its executable and command line match this viewer;
    • the PID file was not replaced or modified by another user.
  6. Prefer a graceful viewer-specific shutdown endpoint authenticated with a random local token over sending a generic signal.
  7. If process termination remains necessary, display the identified process and require explicit confirmation in interactive use.
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
Findings (22)

Tp4

High
Category
MCP Tool Poisoning
Confidence
96% confidence
Finding

The packaged .skill creation and validation steps are not reflected in the user-facing description. While packaging itself is not inherently dangerous, hidden archive creation and disk writes are side effects that expand the skill's behavior beyond what a user would infer from the metadata.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The packaged .skill creation and validation steps are not reflected in the user-facing description. While packaging itself is not inherently dangerous, hidden archive creation and disk writes are side effects that expand the skill's behavior beyond what a user would infer from the metadata.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The packaged .skill creation and validation steps are not reflected in the user-facing description. While packaging itself is not inherently dangerous, hidden archive creation and disk writes are side effects that expand the skill's behavior beyond what a user would infer from the metadata.

Content

No source excerpt is available for this finding.

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 364)May include surrounding context.

md
1. Read the template from `assets/eval_review.html`

Env Variable Harvesting

High
Category
Data Exfiltration
Confidence
70% confidence
Finding

Code enumerates, copies, or searches environment variables for secrets. Bulk environment access can collect credentials unrelated to the skill's stated purpose.

Content

Scanner excerpt · scripts/improve_description.py (reported line 33)May include surrounding context.

python
# Remove CLAUDECODE env var to allow nesting claude -p inside a
    # Claude Code session. The guard is for interactive terminal conflicts;
    # programmatic subprocess usage is safe. Same pattern as run_eval.py.
    env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

    result = subprocess.run(
        cmd,

Env Variable Harvesting

High
Category
Data Exfiltration
Confidence
70% confidence
Finding

Code enumerates, copies, or searches environment variables for secrets. Bulk environment access can collect credentials unrelated to the skill's stated purpose.

Content

Scanner excerpt · scripts/run_eval.py (reported line 83)May include surrounding context.

python
# Remove CLAUDECODE env var to allow nesting claude -p inside a
    # Claude Code session. The guard is for interactive terminal conflicts;
    # programmatic subprocess usage is safe. Same pattern as run_eval.py.
    env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

    result = subprocess.run(
        cmd,

Agent Config Directory Access

High
Category
Agent Snooping
Confidence
85% confidence
Finding

The script automatically discovers a parent project root by locating a .claude directory, then writes a temporary command file into that project's agent configuration area. In a shared or sensitive workspace, this can modify the active Claude command set outside the intended skill directory and influence subsequent agent behavior or evaluations.

Content

Scanner excerpt · scripts/run_eval.py (reported line 23)May include surrounding context.

python
def find_project_root() -> Path:
    """Find the project root by walking up from cwd looking for .claude/.

    Mimics how Claude Code discovers its project root, so the command file
    we create ends up where claude -p will look for it.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
95% confidence
Finding

The skill explicitly instructs the agent to read and write files, execute shell commands, launch scripts, package archives, and manipulate local workspaces, but the frontmatter declares no allowed-tools or equivalent scope. That creates an unnecessarily broad trust boundary: if the skill is invoked, the agent may use powerful capabilities without an explicit least-privilege declaration, increasing the risk of unintended filesystem changes or command execution.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The guidance to make descriptions 'pushy' and trigger on broad adjacent concepts encourages overbroad activation. In practice, this can cause the skill to be invoked in contexts where its shell/file/server behaviors are unnecessary, increasing the chance of accidental side effects and privilege use on unrelated user tasks.

Content

No source excerpt is available for this finding.

Session Persistence

Medium
Category
Rogue Agent
Confidence
83% confidence
Finding

Using nohup to launch the review generator in the background creates a process that persists beyond the immediate interaction, and the skill later instructs killing it via PID management. Background persistence increases the chance of orphaned services, stale review data exposure, or accidental interference with other local processes if lifecycle handling goes wrong.

Content

Scanner excerpt · SKILL.md (reported line 238)May include surrounding context.

  1. Launch the viewer with both qualitative outputs and quantitative data:
    bash
    nohup python <skill-creator-path>/eval-viewer/generate_review.py \
      <workspace>/iteration-N \
      --skill-name "my-skill" \
      --benchmark <workspace>/iteration-N/benchmark.json \
    

Session Persistence

Medium
Category
Rogue Agent
Confidence
88% confidence
Finding

The skill recommends installation locations such as ~/.cursor/skills/skill-name/, which creates persistent availability across future sessions. Persistent installation is risky for a high-capability skill because any overbroad trigger logic or hidden side effects can continue affecting unrelated future interactions long after the original task is finished.

Content

Scanner excerpt · SKILL.md (reported line 476)May include surrounding context.

md
| Personal | `~/.cursor/skills/skill-name/` | All projects |
| Project | `.cursor/skills/skill-name/` | Repository-shared |

**Never** create skills in `~/.cursor/skills-cursor/` — reserved for system built-ins.

### Frontmatter Fields
| Field | Requirements | Purpose |

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

Lines L077-L085 explicitly direct the agent to produce actionable improvement suggestions for the loser skill. Later, lines L189-L190 and L270-L271 state that for benchmark analysis the analyzer's purpose is to surface patterns and anomalies, not suggest skill improvements. These instructions actively conflict and could cause the skill to behave contrary to its own documented intent depending on which section is followed.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
89% confidence
Finding

The skill defines two materially different tasks in one markdown file without a strong activation boundary, which can cause an agent to apply the wrong instruction set to the current input. In practice, this can lead to benchmark runs producing unauthorized improvement recommendations or post-hoc analyses omitting required suggestions, reducing reliability and creating opportunities for prompt-confusion attacks.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The script will enumerate PIDs listening on an arbitrary user-chosen port and send SIGTERM to them before starting its own server. That creates an unintended local denial-of-service capability unrelated to simply viewing eval results, and it can disrupt unrelated developer tools or services without verifying ownership or obtaining confirmation.

Content

No source excerpt is available for this finding.

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · eval-viewer/generate_review.py (reported line 291)May include surrounding context.

python
def _kill_port(port: int) -> None:
    """Kill any process listening on the given port."""
    try:
        result = subprocess.run(
            ["lsof", "-ti", f":{port}"],
            capture_output=True, text=True, timeout=5,
        )

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
98% confidence
Finding

At startup, the program unconditionally calls _kill_port(port), so simply running the viewer can terminate any process currently bound to the selected port. In the context of a skill-creation tool, this is more dangerous because users may run it in active development environments where important local services commonly occupy ports, causing avoidable service interruption or data loss.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

This HTML/JS code auto-saves reviewer feedback by POSTing it to /api/feedback, which is a network transmission of user-provided data. Although the status text says "Saved" after the fact, there is no prior warning or explicit disclosure to the user that their typed feedback will be sent to a server automatically.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

The final submit action POSTs all reviews and timestamps to /api/feedback, which is a network operation involving user-entered content. The button label and completion dialog mention saving, but they do not clearly disclose before submission that the data is being sent to a server endpoint.

Content

No source excerpt is available for this finding.

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · scripts/improve_description.py (reported line 35)May include surrounding context.

python
# programmatic subprocess usage is safe. Same pattern as run_eval.py.
    env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

    result = subprocess.run(
        cmd,
        input=prompt,
        capture_output=True,

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · scripts/run_eval.py (reported line 85)May include surrounding context.

python
# programmatic subprocess usage is safe.
        env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

        process = subprocess.Popen(
            cmd,
            stdout=subprocess.PIPE,
            stderr=subprocess.DEVNULL,

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
88% confidence
Finding

The agent is instructed to write grading.json to a sibling path outside outputs_dir, which is a filesystem side effect not clearly surfaced to the user in the skill description or guarded by path constraints. While expected for a grader workflow, this can still overwrite or create files in adjacent directories if outputs_dir is attacker-controlled or unexpectedly resolved, making it a real but low-severity integrity issue.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Low
Category
Not specified by scanner
Confidence
88% confidence
Finding

The manifest describes functionality around skill creation, editing, evaluation, benchmarking, and description optimization. Automatically launching the user's web browser is not necessary to perform those tasks and introduces an extra capability that affects the user's local environment rather than the skill artifact itself.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.