Back to skill

Security audit

Skill Creator

Security checks for vulnerabilities and agentic risk

Overview

The skill has a legitimate skill-building purpose, but it needs review because its bundled viewer can mishandle untrusted evaluation files and can stop other local apps when it starts.

Install only if you are comfortable with a broad skill-development utility that creates and modifies skills, runs subagents and helper scripts, touches .claude/commands for trigger tests, and may send skill/eval content to Anthropic during description optimization. Before using the bundled viewer on untrusted outputs, fix or avoid the unsafe HTML embedding/rendering paths, remove or gate the automatic port-kill behavior, avoid running it with elevated privileges, and keep evaluation workspaces isolated.

Vulnerability Patterns
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
Findings (3)

T09 · Insecure Skill Coding Practices

Error
Location
eval-viewer/generate_review.py:275
Finding

Stored JavaScript Injection in Generated Evaluation Viewer

Content
View full analysis
dict: """Read a file and return an embedded representation.""" ext = path.suffix.lower() mime = get_mime_type(path) if ext in TEXT_EXTENSIONS: try: content = path.read_text(errors="replace") except OSError: content = "(Error reading file)" return { "name": path.name, "type": "text", "content": content, } ``` ```python embedded = { "skill_name": skill_name, "runs": runs, "previous_feedback": previous_feedback, "previous_outputs": previous_outputs, } if benchmark: embedded["benchmark"] = benchmark data_json = json.dumps(embedded) return template.replace( "/*__EMBEDDED_DATA__*/", f"const EMBEDDED_DATA = {data_json};" ) ``` Relevant unsafe HTML construction in `viewer.html` includes: ```javascript let html = ""; html += "

Benchmark Results

"; html += "

"; if (metadata.skill_name) html += "" + escapeHtml(metadata.skill_name) + " — "; if (metadata.timestamp) html += metadata.timestamp + " — "; if (metadata.evals_run) html += "Evals: " + metadata.evals_run.join(", ") + " — "; html += (metadata.runs_per_configuration || "?") + " runs per configuration"; html += "

"; ``` ```javascript container.innerHTML = html; ``` ### Technical Analysis The review generator recursively reads files produced by evaluated skills and embeds their contents into an executable `
Remediation
View remediation
``` 3. Before embedding JSON into HTML, escape at least `<`, `>`, `&`, U+2028, and U+2029. Parse the element's text using `JSON.parse`. 4. Prefer serving the data from a local JSON endpoint with an explicit `application/json` content type. 5. Replace dynamically assembled HTML and `innerHTML` assignments with DOM construction and `textContent`. 6. Where HTML templates are unavoidable, apply contextual HTML escaping to every interpolated string, including timestamps, evaluation names, configuration values, deltas, notes, and evidence. 7. Add a restrictive Content Security Policy, such as a nonce-based `script-src`, and restrict `connect-src` to the loopback origin. 8. Add regression tests using output and benchmark values containing script-closing sequences, HTML tags, event handlers, quotes, and Unicode line separators. 9. Treat every evaluated output and benchmark file as untrusted, even when it was generated locally. ]]>

T05 · Unauthorized Access and Privilege Escalation

Warning
Location
eval-viewer/generate_review.py:286
Finding

Viewer Startup Can Terminate Unrelated Processes Listening on the Selected Port

Content
View full analysis
None: """Kill any process listening on the given port.""" try: result = subprocess.run( ["lsof", "-ti", f":{port}"], capture_output=True, text=True, timeout=5, ) for pid_str in result.stdout.strip().split("\n"): if pid_str.strip(): try: os.kill(int(pid_str.strip()), signal.SIGTERM) except (ProcessLookupError, ValueError): pass if result.stdout.strip(): time.sleep(0.5) except subprocess.TimeoutExpired: pass except FileNotFoundError: print("Note: lsof not found, cannot check if port is in use", file=sys.stderr) ``` ```python # Kill any existing process on the target port port = args.port _kill_port(port) handler = partial( ReviewHandler, workspace, skill_name, feedback_path, previous, benchmark_path, ) try: server = HTTPServer(("127.0.0.1", port), handler) except OSError: # Port still in use after kill attempt — find a free one server = HTTPServer(("127.0.0.1", 0), handler) port = server.server_address[1] ``` ### Technical Analysis The viewer unconditionally invokes `lsof` for the selected port and sends `SIGTERM` to every PID returned. It does not verify that the process is a previous instance of the evaluation viewer, that the process was started by this project, or that terminating it was authorized by the user. A local viewer only needs to bind to an available loopback port. The code already contains a fallback that asks the operating system to select a free port, demonstrating that terminating an existing listener is unnecessary for the declared functionality. The ope ...[truncated 1621 chars]
Remediation
View remediation

T08 · Insecure Dependencies

Note
Location
eval-viewer/viewer.html:7
Finding

Local Review Pages Load Runtime Resources from Third-Party Origins

Content
View full analysis
``` From `assets/eval_review.html`: ```html ``` From the HTML template in `scripts/generate_report.py`: ```html ``` ### Technical Analysis Opening locally generated review and report pages causes the browser to connect to Google Fonts. The main viewer also loads SheetJS from a third-party CDN at runtime. The SheetJS resource is version-pinned and protected by a Subresource Integrity hash, which substantially limits payload substitution: a modified response should be rejected unless it matches the expected hash. No evidence was found that the dependency is intentionally malicious. Nevertheless, a runtime CDN dependency is not necessary for a local evalua ...[truncated 1454 chars]
Remediation
View remediation
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
Findings (21)

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 374)May include surrounding context.

md
1. Read the template from `assets/eval_review.html`

Context-Inappropriate Capability

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

On startup, the script unconditionally kills whatever local process is bound to the requested port before starting its own server. For a review viewer, that capability is unjustified and can be abused or accidentally triggered to stop unrelated applications, causing denial of service on the local host.

Content

No source excerpt is available for this finding.

Agent Config Directory Access

High
Category
Agent Snooping
Confidence
85% confidence
Finding

Skill reads from agent configuration directories (.claude/, .codex/, .gemini/). These directories may contain API keys, personal settings, and other credentials that the skill has no legitimate need to access.

Content

Scanner excerpt · scripts/run_eval.py (reported line 23)May include surrounding context.

python
def find_project_root() -> Path:
    """Find the project root by walking up from cwd looking for .claude/.

    Mimics how Claude Code discovers its project root, so the command file
    we create ends up where claude -p will look for it.

Env Variable Harvesting

High
Category
Data Exfiltration
Confidence
70% confidence
Finding

Code enumerates, copies, or searches environment variables for secrets. Bulk environment access can collect credentials unrelated to the skill's stated purpose.

Content

Scanner excerpt · scripts/run_eval.py (reported line 83)May include surrounding context.

python
# Remove CLAUDECODE env var to allow nesting claude -p inside a
        # Claude Code session. The guard is for interactive terminal conflicts;
        # programmatic subprocess usage is safe.
        env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

        process = subprocess.Popen(
            cmd,

Session Persistence

Medium
Category
Rogue Agent
Confidence
60% confidence
Finding

Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.

Content

Scanner excerpt · SKILL.md (reported line 3)May include surrounding context.

md
---
name: skill-creator
description: [USM] Create new skills, modify and improve existing skills, measure skill performance, and manage skill distribution across Agents. Use when users want to create a skill from scratch, update or optimize an existing skill, run evals, or when they ask to sync skills, manage the skill hub, or configure which agents can see which skills using the 2-layer 2-dimension architecture.
---

# Skill Creator

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The instruction to make skill descriptions a little bit 'pushy' explicitly biases authors toward broader triggering to compensate for undertriggering. That can cause the wrong skill to activate in unrelated contexts, increasing the chance of unintended file operations, tool use, or workflow execution beyond what the user actually asked for.

Content

No source excerpt is available for this finding.

Session Persistence

Medium
Category
Rogue Agent
Confidence
88% confidence
Finding

Using nohup to launch the review server intentionally detaches the process from the current session so it continues running in the background. In a security-sensitive environment, detached long-lived processes can expose artifacts, consume resources, keep ports open, and outlive user awareness, especially if cleanup depends on a later manual kill.

Content

Scanner excerpt · SKILL.md (reported line 248)May include surrounding context.

  1. Launch the viewer with both qualitative outputs and quantitative data:
    bash
    nohup python <skill-creator-path>/eval-viewer/generate_review.py \
      <workspace>/iteration-N \
      --skill-name "my-skill" \
      --benchmark <workspace>/iteration-N/benchmark.json \
    

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The skill combines two distinct analyzer behaviors across the full file without clearly delimiting when each should apply. This ambiguity can be exploited by prompt context or malformed inputs to steer the agent into the wrong workflow, leading to incorrect file access patterns, invalid outputs, and weakened trust in evaluation results.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The file defines an initial role that writes a structured post-hoc comparison report, then later introduces a benchmark-analysis mode with a different purpose and a different output schema. Without explicit invocation boundaries or separate files, an agent may follow the wrong section, emit the wrong format, or read unintended inputs, causing analysis corruption and unreliable downstream automation.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The skill explicitly instructs the agent to read paths supplied as inputs, including arbitrary files or entire directories, and to write a JSON result file to a specified path. Without constraining allowed locations, this can enable unintended access to sensitive local data and creation or overwriting of files in the agent's workspace, especially if an attacker controls the path arguments.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

The helper is described as handling port conflicts, but its actual behavior is to terminate any process listening on the selected port. That mismatch is dangerous because users may run the tool expecting harmless setup behavior, while it can disrupt unrelated services or development processes on the machine.

Content

No source excerpt is available for this finding.

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
92% confidence
Finding

The code invokes an external system utility to identify processes on a port, which is not command injection here because arguments are passed as a list, but it is part of a workflow that can terminate unrelated local processes. In the context of a simple review viewer, this creates unnecessary host-side side effects and expands the script's ability beyond serving content.

Content

Scanner excerpt · eval-viewer/generate_review.py (reported line 291)May include surrounding context.

python
def _kill_port(port: int) -> None:
    """Kill any process listening on the given port."""
    try:
        result = subprocess.run(
            ["lsof", "-ti", f":{port}"],
            capture_output=True, text=True, timeout=5,
        )

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The script terminates any process using the chosen port without warning or consent, creating a local denial-of-service risk. The skill context makes this more dangerous because a benign-seeming viewer should not silently perform destructive host-management actions unrelated to its core function.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

The XLSX renderer converts spreadsheet content to HTML and inserts it via innerHTML. If workbook cell content is attacker-controlled and SheetJS output is not strictly sanitized for all constructs, this can enable DOM-based XSS in the reviewer's browser, especially since this viewer is explicitly designed to open and inspect untrusted outputs.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The script sends full skill content, evaluation queries/results, and prior attempt history to Anthropic in a remote API call, but this file contains no consent gate, redaction, or disclosure mechanism. If those inputs contain proprietary prompts, sensitive test data, or user-derived content, they are exposed to a third party unnecessarily, which creates a real data-leakage and compliance risk.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The transcript logging captures and persists the full prompt, model 'thinking', responses, and parsed descriptions to disk. Because the prompt includes skill content, eval queries, and history, the log file can become a durable local copy of sensitive data and internal reasoning artifacts, increasing exposure through filesystem access, backups, or accidental sharing.

Content

No source excerpt is available for this finding.

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · scripts/run_eval.py (reported line 85)May include surrounding context.

python
# programmatic subprocess usage is safe.
        env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

        process = subprocess.Popen(
            cmd,
            stdout=subprocess.PIPE,
            stderr=subprocess.DEVNULL,

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
82% confidence
Finding

This markdown skill description explicitly directs the agent to save results to {outputs_dir}/../grading.json, which is a file write affecting user data or workspace contents. The document does not include any warning or disclosure that the skill will create or overwrite this file.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
82% confidence
Finding

This code writes reviewer-submitted data directly to feedback.json, which affects user data on disk. While the module docstring mentions auto-saving, the write operation itself has no confirmation prompt or user-facing disclosure in the handler, so a user interacting only through the served page may not receive a clear warning at the point of modification.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Low
Category
Not specified by scanner
Confidence
76% confidence
Finding

The init comment at L0658-L0661 says saved feedback is loaded from the server under certain conditions, framing persistence around server state, while later code explicitly supports a static/offline mode and downloads feedback.json locally on final submit at L1050-L1059. This is not a security bug by itself, but the inline documentation presents the persistence model more narrowly than the implemented behavior.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
90% confidence
Finding

This code opens the generated HTML report in the user's default web browser via webbrowser.open(...) without prompting the user or clearly disclosing that an external application will be launched. For a code file, subprocess-like or system-affecting actions should have some visible disclosure, and the surrounding help text does not explicitly warn that a browser will open automatically.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.