Back to skill

Security audit

Skill Creator

Security checks for vulnerabilities and agentic risk

Overview

This skill mostly does what it says, but its review and evaluation tools include unsafe local browser and process behavior that users should review carefully before installing.

Install only if you are comfortable with a skill that runs local evaluation scripts, calls the claude CLI, writes temporary .claude command files, opens local review HTML, and may start a localhost server. Prefer static review output for untrusted evaluations, avoid opening generated review pages from adversarial skill outputs, and check the chosen viewer port first because the bundled server may terminate another local process using it.

Vulnerability Patterns
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (3)

T09 · Insecure Skill Coding Practices

Error
Location
eval-viewer/generate_review.py:149
Finding

Stored Script Injection Through Embedded Evaluation Outputs

Content
View full analysis
dict: """Read a file and return an embedded representation.""" ext = path.suffix.lower() mime = get_mime_type(path) if ext in TEXT_EXTENSIONS: try: content = path.read_text(errors="replace") except OSError: content = "(Error reading file)" return { "name": path.name, "type": "text", "content": content, } elif ext in IMAGE_EXTENSIONS: try: raw = path.read_bytes() b64 = base64.b64encode(raw).decode("ascii") except OSError: return {"name": path.name, "type": "error", "content": "(Error reading file)"} return { "name": path.name, "type": "image", "mime": mime, "data_uri": f"data:{mime};base64,{b64}", } elif ext == ".pdf": try: raw = path.read_bytes() b64 = base64.b64encode(raw).decode("ascii") except OSError: return {"name": path.name, "type": "error", "content": "(Error reading file)"} return { "name": path.name, "type": "pdf", "data_uri": f"data:{mime};base64,{b64}", } elif ext == ".xlsx": try: raw = path.read_bytes() b64 = base64.b64encode(raw).decode("ascii") except OSError: return {"name": path.name, "type": "error", "content": "(Error reading file)"} return { "name": path.name, "type": "xlsx", "data_b64": b64, } else: # Binary / unknown — base64 download link try: raw = path.read_bytes() b ...[truncated 3302 chars]
Remediation
View remediation
SAFE_JSON_DATA ``` 3. Before embedding JSON into HTML, encode HTML-significant characters, at minimum: ```python data_json = json.dumps(embedded) data_json = ( data_json .replace("<", "\\u003c") .replace(">", "\\u003e") .replace("&", "\\u0026") ) ``` 4. Parse the non-executable element at runtime: ```javascript const EMBEDDED_DATA = JSON.parse( document.getElementById("embedded-data").textContent ); ``` 5. Add a restrictive Content Security Policy that disallows arbitrary inline scripts and limits outbound connections, for example by using a nonce-bearing local script and `connect-src 'self'`. 6. Prefer serving downloadable output files through validated local endpoints instead of embedding every file into the page. 7. Add regression tests using payloads containing ``, mixed-case ``, HTML comments, Unicode separators, and malicious SVG/HTML content. ]]>

T09 · Insecure Skill Coding Practices

Error
Location
assets/eval_review.html:5
Finding

Script and HTML Injection in the Eval-Set Review Template

Content
View full analysis
.html`) and open it: `open /tmp/eval_review_.html` ``` The placeholders occur in multiple HTML and JavaScript contexts: ```html Eval Set Review - __SKILL_NAME_PLACEHOLDER__ ``` ```html

Eval Set Review: __SKILL_NAME_PLACEHOLDER__

Current description: __SKILL_DESCRIPTION_PLACEHOLDER__

``` ```html ` can terminate the surrounding script element because JSON escaping does not automatically neutralize HTML end tags. The name and description placeholders are also inserted as raw markup. Existing vali ...[truncated 1874 chars]
Remediation
View remediation
`, and `&` in serialized JSON, or store JSON in a non-executable `application/json` element. 4. Populate Skill names and descriptions through DOM `textContent` rather than raw template replacement: ```html ``` ```javascript document.getElementById("skill-name").textContent = metadata.skillName; document.getElementById("skill-desc").textContent = metadata.description; ``` 5. Require `quick_validate.py` or equivalent validation before generating a review page from an existing Skill. 6. Use a strict Content Security Policy with no unrestricted inline-script execution and a restrictive `connect-src`. 7. Add security tests covering hostile names, descriptions, and queries, including closing script tags and malformed HTML. ]]>

T05 · Unauthorized Access and Privilege Escalation

Warning
Location
eval-viewer/generate_review.py:286
Finding

Viewer Startup Terminates Unrelated Processes Using the Configured Port

Content
View full analysis
None: """Kill any process listening on the given port.""" try: result = subprocess.run( ["lsof", "-ti", f":{port}"], capture_output=True, text=True, timeout=5, ) for pid_str in result.stdout.strip().split("\n"): if pid_str.strip(): try: os.kill(int(pid_str.strip()), signal.SIGTERM) except (ProcessLookupError, ValueError): pass if result.stdout.strip(): time.sleep(0.5) except subprocess.TimeoutExpired: pass except FileNotFoundError: print("Note: lsof not found, cannot check if port is in use", file=sys.stderr) ``` The function is invoked automatically during viewer startup: ```python # Kill any existing process on the target port port = args.port _kill_port(port) handler = partial( ReviewHandler, workspace, skill_name, feedback_path, previous, benchmark_path, ) try: server = HTTPServer(("127.0.0.1", port), handler) except OSError: # Port still in use after kill attempt — find a free one server = HTTPServer(("127.0.0.1", 0), handler) port = server.server_address[1] ``` ### Technical Analysis The viewer invokes `lsof` to obtain every process listening on the requested port and sends `SIGTERM` to each returned process. It does not verify that a process is: - A previous instance of this viewer. - Owned or launched by the current workflow. - Safe to terminate. - In a state where termination will not cause data loss. The process is killed before the viewer attempts to bind the port. This behavior is unnecessary because the code already contains a safe fallback that binds to ...[truncated 1441 chars]
Remediation
View remediation
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
Findings (16)

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The declared description focuses on skill authoring, editing, optimization, evaluation, and benchmarking. However, the supplied code does not implement any of those core functions. Its primary purpose is packaging an existing skill directory into a .skill file after basic validation. That is a materially different capability from creating/modifying skills or measuring skill performance. While validation is loosely related to skill maintenance, the main behavior is distribution packaging, which is undeclared and not represented in the description.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The declared description presents a broad skill-development and evaluation capability suite, including creation, editing, optimization, evaluation, benchmarking, and trigger-description tuning. The actual code only performs basic static validation of a skill's SKILL.md frontmatter and related formatting constraints. While validation could be a supporting piece of a larger skill-authoring workflow, this code chunk's primary behavior is materially narrower and different from the declared purpose. There is no evidence of skill generation, modification, performance measurement, or optimization logic.

Content

No source excerpt is available for this finding.

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 364)May include surrounding context.

md
1. Read the template from `assets/eval_review.html`

Env Variable Harvesting

High
Category
Data Exfiltration
Confidence
92% confidence
Finding

The code copies nearly the entire parent process environment into the child claude subprocess, removing only CLAUDECODE. In a skill-creation workflow, the prompt includes skill content, eval history, and other potentially adversarial inputs; if the external CLI or any downstream tool/plugin can access inherited environment variables, secrets such as API keys, tokens, or internal endpoints may be exposed beyond the minimum necessary scope.

Content

Scanner excerpt · scripts/improve_description.py (reported line 33)May include surrounding context.

python
# Remove CLAUDECODE env var to allow nesting claude -p inside a
    # Claude Code session. The guard is for interactive terminal conflicts;
    # programmatic subprocess usage is safe. Same pattern as run_eval.py.
    env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

    result = subprocess.run(
        cmd,

Agent Config Directory Access

High
Category
Agent Snooping
Confidence
87% confidence
Finding

The script intentionally discovers the nearest .claude directory and then writes a temporary command file into .claude/commands, modifying agent configuration in the surrounding project. In a skill context, touching agent config directories is sensitive because it can influence what the downstream agent sees as available commands/skills and could affect unrelated sessions if cleanup fails or if the project root is broader than expected.

Content

Scanner excerpt · scripts/run_eval.py (reported line 23)May include surrounding context.

python
def find_project_root() -> Path:
    """Find the project root by walking up from cwd looking for .claude/.

    Mimics how Claude Code discovers its project root, so the command file
    we create ends up where claude -p will look for it.

Env Variable Harvesting

High
Category
Data Exfiltration
Confidence
70% confidence
Finding

Code enumerates, copies, or searches environment variables for secrets. Bulk environment access can collect credentials unrelated to the skill's stated purpose.

Content

Scanner excerpt · scripts/run_eval.py (reported line 83)May include surrounding context.

python
# Remove CLAUDECODE env var to allow nesting claude -p inside a
        # Claude Code session. The guard is for interactive terminal conflicts;
        # programmatic subprocess usage is safe.
        env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

        process = subprocess.Popen(
            cmd,

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
91% confidence
Finding

The skill instructs the agent to read/write files, inspect environment context, and execute shell commands, but it declares no explicit tool scope or allowed-tools boundary. That creates an overprivileged skill surface: if invoked in a permissive environment, the agent may perform filesystem and command actions broader than a user expects, increasing the chance of unintended file modification, data exposure, or command execution.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The guidance to make descriptions 'pushy' and trigger on broad related mentions encourages overbroad invocation criteria. In practice, that can cause the skill to activate for loosely related requests, exposing users to unnecessary file, shell, eval, and packaging workflows and increasing the chance of unintended side effects or interference with more appropriate skills.

Content

No source excerpt is available for this finding.

Session Persistence

Medium
Category
Rogue Agent
Confidence
79% confidence
Finding

Using nohup to launch the review viewer creates a persistent background process that can outlive the immediate task and continue serving files or consuming resources after the user interaction ends. In shared or long-lived environments, this increases the risk of stale services, accidental exposure of workspace contents, and poor process hygiene, especially since cleanup depends on later manual termination.

Content

Scanner excerpt · SKILL.md (reported line 238)May include surrounding context.

  1. Launch the viewer with both qualitative outputs and quantitative data:
    bash
    nohup python <skill-creator-path>/eval-viewer/generate_review.py \
      <workspace>/iteration-N \
      --skill-name "my-skill" \
      --benchmark <workspace>/iteration-N/benchmark.json \
    

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The file defines two distinct operating modes in one agent ('post-hoc skill comparison' and 'benchmark analysis') and says 'When analyzing benchmark results...' without a clear, enforceable trigger boundary or selection rule. In an agentic system that routes by natural-language descriptions, this ambiguity can cause the analyzer to activate in the wrong mode, leading to incorrect file access, wrong output schema, or leakage of comparison/transcript data into benchmark-note workflows.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

This code gives the eval viewer an unnecessary destructive capability: it kills any local process listening on the requested port, regardless of ownership or relationship to the tool. That exceeds the stated purpose of generating and serving a review page and can cause denial of service against unrelated local applications, especially if the port is chosen intentionally or by mistake.

Content

No source excerpt is available for this finding.

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
92% confidence
Finding

The subprocess invocation itself is not command-injection prone because it uses an argument list and the port is parsed as an integer, but it is part of functionality that enumerates and enables termination of whatever process is bound to a user-selected port. In a local developer environment, this can disrupt unrelated services or tooling and becomes more concerning because the script automatically performs the action before attempting to bind its own server.

Content

Scanner excerpt · eval-viewer/generate_review.py (reported line 291)May include surrounding context.

python
def _kill_port(port: int) -> None:
    """Kill any process listening on the given port."""
    try:
        result = subprocess.run(
            ["lsof", "-ti", f":{port}"],
            capture_output=True, text=True, timeout=5,
        )

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The program invokes port-clearing logic automatically with no immediate warning or confirmation at the execution point, so users may not realize it can kill another local service before it happens. Hidden destructive behavior increases the chance of accidental service interruption and makes the capability riskier in a skill whose expected function is only to view evaluation results.

Content

No source excerpt is available for this finding.

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · scripts/improve_description.py (reported line 35)May include surrounding context.

python
# programmatic subprocess usage is safe. Same pattern as run_eval.py.
    env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

    result = subprocess.run(
        cmd,
        input=prompt,
        capture_output=True,

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · scripts/run_eval.py (reported line 85)May include surrounding context.

python
# programmatic subprocess usage is safe.
        env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

        process = subprocess.Popen(
            cmd,
            stdout=subprocess.PIPE,
            stderr=subprocess.DEVNULL,

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
79% confidence
Finding

The markdown instructs the agent to save results to a sibling path (grading.json), which is a file write that can affect user data or workspace state. The skill description provides no warning that it will create or overwrite a file, nor any confirmation or caution about that behavior.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.