Back to skill

Security audit

Skill Creator

Security checks for vulnerabilities and agentic risk

Overview

The skill’s purpose is coherent, but its review viewer has unsafe local-server behavior and an HTML injection flaw that could expose evaluation data.

Review carefully before installing. Use the static viewer mode where possible, avoid opening review pages for outputs from untrusted skills until the HTML embedding is fixed, and do not run the local viewer on a port used by other services. Expect the skill to write evaluation artifacts, use claude -p with your current Claude Code auth, temporarily touch .claude/commands during trigger tests, and require manual cleanup of background viewer processes.

Vulnerability Patterns
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (2)

T09 · Insecure Skill Coding Practices

Error
Location
eval-viewer/generate_review.py:263
Finding

Stored Script Injection Through Unescaped Embedded Evaluation Data

Content
View full analysis
str: """Generate the complete standalone HTML page with embedded data.""" template_path = Path(__file__).parent / "viewer.html" template = template_path.read_text() # Build previous_feedback and previous_outputs maps for the template previous_feedback: dict[str, str] = {} previous_outputs: dict[str, list[dict]] = {} if previous: for run_id, data in previous.items(): if data.get("feedback"): previous_feedback[run_id] = data["feedback"] if data.get("outputs"): previous_outputs[run_id] = data["outputs"] embedded = { "skill_name": skill_name, "runs": runs, "previous_feedback": previous_feedback, "previous_outputs": previous_outputs, } if benchmark: embedded["benchmark"] = benchmark data_json = json.dumps(embedded) return template.replace("/*__EMBEDDED_DATA__*/", f"const EMBEDDED_DATA = {data_json};") ``` The resulting data is inserted directly into this executable script element: ```html ` terminates the surround ...[truncated 2244 chars]
Remediation
View remediation
", "\\u003e") .replace("\u2028", "\\u2028") .replace("\u2029", "\\u2029") ) ``` 2. Prefer embedding the object in a non-executable JSON container: ```html ``` Then parse its text explicitly: ```javascript const EMBEDDED_DATA = JSON.parse( document.getElementById("embedded-data").textContent ); ``` 3. Apply a restrictive Content Security Policy. Avoid inline scripts or authorize only a fixed script hash or nonce. Restrict outbound connections with `connect-src 'self'` and other resource directives. 4. Treat every prompt, output file, filename, grading field, benchmark field, and feedback value as attacker-controlled input. 5. Add regression tests that generate and open viewers containing: - `` - HTML tags and event handlers - U+2028 and U+2029 characters - Strings containing `<`, `>`, and `&` 6. Consider rendering the review UI from a fixed application page and loading evaluation data through a JSON endpoint with the correct `application/json` content type, rather than injecting data into executable source code. ]]>

T05 · Unauthorized Access and Privilege Escalation

Warning
Location
eval-viewer/generate_review.py:287
Finding

Viewer Startup Unconditionally Terminates Unrelated Processes Using the Requested Port

Content
View full analysis
None: """Kill any process listening on the given port.""" try: result = subprocess.run( ["lsof", "-ti", f":{port}"], capture_output=True, text=True, timeout=5, ) for pid_str in result.stdout.strip().split("\n"): if pid_str.strip(): try: os.kill(int(pid_str.strip()), signal.SIGTERM) except (ProcessLookupError, ValueError): pass if result.stdout.strip(): time.sleep(0.5) except subprocess.TimeoutExpired: pass except FileNotFoundError: print("Note: lsof not found, cannot check if port is in use", file=sys.stderr) ``` The function is invoked unconditionally before the viewer binds its server socket: ```python # Kill any existing process on the target port port = args.port _kill_port(port) ``` ### Technical Analysis The viewer uses `lsof` to obtain every process identifier listening on the requested port and sends `SIGTERM` to each process. It does not verify that a target process: - Was previously created by this application. - Is another instance of the evaluation viewer. - Belongs to the current workspace. - Can be safely terminated. - Has received user confirmation for termination. This operation is unnecessary for the declared review functionality. The implementation already contains a safer fallback that binds to port `0` and obtains an available ephemeral port if the requested port cannot be used. Although `subprocess.run()` does not use a shell and is not vulnerable to shell-command injection here, the process-management behavior breaks least-privilege boundari ...[truncated 1577 chars]
Remediation
View remediation
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
Findings (24)

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding

The frontmatter describes a skill-creation/evaluation assistant, but the body also directs launching a local review server, embedding run outputs into HTML, loading prior feedback, and killing processes. Those extra behaviors materially change the security posture because they expose local data and process-control actions that a user may not reasonably expect from the description alone.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The frontmatter describes a skill-creation/evaluation assistant, but the body also directs launching a local review server, embedding run outputs into HTML, loading prior feedback, and killing processes. Those extra behaviors materially change the security posture because they expose local data and process-control actions that a user may not reasonably expect from the description alone.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
96% confidence
Finding

The frontmatter describes a skill-creation/evaluation assistant, but the body also directs launching a local review server, embedding run outputs into HTML, loading prior feedback, and killing processes. Those extra behaviors materially change the security posture because they expose local data and process-control actions that a user may not reasonably expect from the description alone.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The frontmatter describes a skill-creation/evaluation assistant, but the body also directs launching a local review server, embedding run outputs into HTML, loading prior feedback, and killing processes. Those extra behaviors materially change the security posture because they expose local data and process-control actions that a user may not reasonably expect from the description alone.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The skill explicitly advises authors to make descriptions 'pushy' to counter undertriggering, encouraging broad activation on adjacent requests. Overbroad trigger descriptions can cause unintended invocation in unrelated contexts, pulling in powerful file/shell workflows and increasing the chance of unsafe actions being applied to the wrong task.

Content

No source excerpt is available for this finding.

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 364)May include surrounding context.

md
1. Read the template from `assets/eval_review.html`

Context-Inappropriate Capability

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

The script will unconditionally discover and send SIGTERM to any local process listening on the chosen port before starting its own server. In a skill-creation/eval workflow, this can disrupt unrelated developer services, test infrastructure, or security-sensitive local applications, and the action occurs automatically with no ownership check or confirmation.

Content

No source excerpt is available for this finding.

Env Variable Harvesting

High
Category
Data Exfiltration
Confidence
70% confidence
Finding

Code enumerates, copies, or searches environment variables for secrets. Bulk environment access can collect credentials unrelated to the skill's stated purpose.

Content

Scanner excerpt · scripts/improve_description.py (reported line 33)May include surrounding context.

python
# Remove CLAUDECODE env var to allow nesting claude -p inside a
    # Claude Code session. The guard is for interactive terminal conflicts;
    # programmatic subprocess usage is safe. Same pattern as run_eval.py.
    env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

    result = subprocess.run(
        cmd,

Env Variable Harvesting

High
Category
Data Exfiltration
Confidence
70% confidence
Finding

Code enumerates, copies, or searches environment variables for secrets. Bulk environment access can collect credentials unrelated to the skill's stated purpose.

Content

Scanner excerpt · scripts/run_eval.py (reported line 83)May include surrounding context.

python
# Remove CLAUDECODE env var to allow nesting claude -p inside a
    # Claude Code session. The guard is for interactive terminal conflicts;
    # programmatic subprocess usage is safe. Same pattern as run_eval.py.
    env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

    result = subprocess.run(
        cmd,

Agent Config Directory Access

High
Category
Agent Snooping
Confidence
85% confidence
Finding

Skill reads from agent configuration directories (.claude/, .codex/, .gemini/). These directories may contain API keys, personal settings, and other credentials that the skill has no legitimate need to access.

Content

Scanner excerpt · scripts/run_eval.py (reported line 23)May include surrounding context.

python
def find_project_root() -> Path:
    """Find the project root by walking up from cwd looking for .claude/.

    Mimics how Claude Code discovers its project root, so the command file
    we create ends up where claude -p will look for it.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
91% confidence
Finding

The skill instructs the agent to read/write files, execute shell commands, inspect environment-dependent tooling, and manage processes, but the skill metadata declares no explicit tool scope or permissions boundary. In a skill that automates evaluation workflows and handles arbitrary user-provided skill paths and workspaces, missing scope constraints increases the blast radius of prompt mistakes or malicious task inputs.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The instructions start a local viewer/server, potentially in the background, and open review content without requiring an explicit user-facing warning or consent step. Because the viewer serves local run outputs and feedback artifacts, this can expose sensitive workspace data and create persistent processes the user did not knowingly approve.

Content

No source excerpt is available for this finding.

Session Persistence

Medium
Category
Rogue Agent
Confidence
92% confidence
Finding

Using nohup to launch the review tool creates a background process that persists beyond the immediate interaction and may continue serving local review data until manually stopped. In a skill that handles arbitrary evaluation artifacts, this persistence increases the risk of unnoticed data exposure, port conflicts, and residual processes on the host.

Content

Scanner excerpt · SKILL.md (reported line 238)May include surrounding context.

  1. Launch the viewer with both qualitative outputs and quantitative data:
    bash
    nohup python <skill-creator-path>/eval-viewer/generate_review.py \
      <workspace>/iteration-N \
      --skill-name "my-skill" \
      --benchmark <workspace>/iteration-N/benchmark.json \
    

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

Lines L189-L190 state that when analyzing benchmark results, the analyzer's purpose is to surface patterns and anomalies and not suggest skill improvements. However, the same file earlier instructs the analyzer to generate improvement suggestions for the losing skill (L77-L85, L129-L147), and the document does not define an explicit mechanism that cleanly gates one mode versus the other. This creates an intent-level contradiction in the documentation that could lead the agent to perform the wrong kind of analysis.

Content

No source excerpt is available for this finding.

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · eval-viewer/generate_review.py (reported line 291)May include surrounding context.

python
def _kill_port(port: int) -> None:
    """Kill any process listening on the given port."""
    try:
        result = subprocess.run(
            ["lsof", "-ti", f":{port}"],
            capture_output=True, text=True, timeout=5,
        )

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · scripts/improve_description.py (reported line 35)May include surrounding context.

python
# programmatic subprocess usage is safe. Same pattern as run_eval.py.
    env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

    result = subprocess.run(
        cmd,
        input=prompt,
        capture_output=True,

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The script sends full skill content, eval results, and prior history to an external claude -p process, which may forward that data to a remote model service. If SKILL.md, eval queries, or history contain secrets, proprietary code, or sensitive prompts, this causes unintended data disclosure without any in-file notice, consent gate, or redaction step.

Content

No source excerpt is available for this finding.

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · scripts/run_eval.py (reported line 85)May include surrounding context.

python
# programmatic subprocess usage is safe.
        env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

        process = subprocess.Popen(
            cmd,
            stdout=subprocess.PIPE,
            stderr=subprocess.DEVNULL,

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
82% confidence
Finding

This markdown skill instructs the agent to save structured analysis to output_path, which is a file write affecting user or system data. The description does not include any warning, confirmation, or disclosure that the skill will write output files, even though SQP-2 applies to markdown files when behaviors affecting data or system integrity are not warned about.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
77% confidence
Finding

This markdown skill instructs the agent to save results to a grading.json file, which is a file write affecting user or system data. The document gives the write location but does not include any warning, disclosure, or note to the user that the skill will create or overwrite that file.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
94% confidence
Finding

The root HTML element sets lang="en", which imposes an English-language locale in the UI. Under the policy, forcing a specific language without user opt-in or documented regional justification is a natural-language policy concern.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
78% confidence
Finding

This code file performs file writes to benchmark.json and benchmark.md, which is a safety-relevant operation under the audit criteria. Although the script's purpose is aggregation, there is no confirmation prompt or explicit disclosure that the default output path may overwrite existing files in the benchmark directory.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
89% confidence
Finding

This code performs a file write and launches the user's browser, but there is no confirmation prompt and no user-facing disclosure at the point of action beyond the CLI flag name. For a code file, SQP-2 applies to file writes and subprocess-like external actions when they lack visible warning or explanation in code comments or runtime messaging.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
80% confidence
Finding

The script creates directories and writes results.json, and later report.html, containing run history and evaluation data. Although this is gated by an argument, the code does not visibly warn the user that evaluation outputs and logs will be persisted to disk, which is the type of file-write disclosure SQP-2 asks for in code files.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.