Back to skill

Security audit

Skill Creator Claude

Security checks for vulnerabilities and agentic risk

Overview

This is a coherent skill-building tool, but its review and packaging helpers can expose local data or disrupt local services if used on untrusted skills.

Review this before installing. Use it only in trusted or isolated workspaces, avoid opening its generated review pages for outputs or eval queries from untrusted skills until the HTML embedding is fixed, inspect skill folders for symlinks before packaging, avoid running the viewer on ports used by other local services, and treat description optimization as sending skill/eval content to Anthropic's API.

Vulnerability Patterns
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (4)

T09 · Insecure Skill Coding Practices

Error
Location
eval-viewer/generate_review.py:270
Finding

Stored Script Injection Through Embedded Evaluation Artifacts

Content
View full analysis
// ---- Embedded data (injected by generate_review.py) ---- /*__EMBEDDED_DATA__*/ // ---- State ---- let feedbackMap = {}; ``` ### Technical Analysis `generate_html()` serializes evaluation prompts, output files, grading information, previous outputs, and previous feedback with `json.dumps()`, then inserts the result directly into an executable ``. The browser recognizes that sequence as the end of the script element even when it appears inside a JavaScript string. An evaluated Skill or other untrusted producer can place a payload such as the following in a text output: ```html ``` When `generate_review.py` reads that output and generates the review page, the payload becomes part of the inline script. Opening or refreshing the viewer executes the injected JavaScript in the viewer's origin. This is particularly dangerous because evaluation outputs are inherently untrusted: the purpose of the viewer is to display files produced by Skills and agents under test. ### Attack Path 1. An attacker supplies or influences a Skill being evaluated. 2. The Skill creates an output file under a discovered `outputs/` directory. 3. The output contains ...[truncated 1359 chars]
Remediation
View remediation
``` Then populate it with HTML-safe JSON and parse its `textContent`: ```javascript const EMBEDDED_DATA = JSON.parse( document.getElementById("embedded-data").textContent ); ``` 3. Before embedding JSON in HTML, escape at least: - `<` as `\u003c` - `>` as `\u003e` - `&` as `\u0026` - U+2028 as `\u2028` - U+2029 as `\u2029` 4. Prefer a vetted templating mechanism that applies context-aware escaping. 5. Add a restrictive Content Security Policy. Ideally, prohibit inline scripts and restrict `connect-src` to `'self'`. 6. Add regression tests using output strings containing: - `` - `

T09 · Insecure Skill Coding Practices

Error
Location
assets/eval_review.html:41
Finding

Script Injection Through Raw Eval-Review Template Substitution

Content
View full analysis

Eval Set Review: __SKILL_NAME_PLACEHOLDER__

Current description: __SKILL_DESCRIPTION_PLACEHOLDER__

+ Add Query Export Eval Set
QueryShould TriggerActions

` sequence in an evaluation query can terminate the script block and int ...[truncated 1892 chars]
Remediation
View remediation
``` ```javascript document.getElementById("skill-name").textContent = metadata.skillName; document.getElementById("skill-desc").textContent = metadata.description; ``` 4. Put evaluation data in an `application/json` element and escape HTML-significant characters before embedding it. 5. If placeholders must remain, use context-specific escaping: - HTML escaping for text-node placeholders. - Script-safe JSON serialization for JavaScript data. 6. Add a CSP that blocks arbitrary inline script execution and restricts outbound connections. 7. Add automated tests with malicious descriptions and queries containing ``, HTML tags, event handlers, quotes, and Unicode line separators. 8. Update `SKILL.md` so agents are never instructed to perform raw textual substitution into HTML. ]]>

T05 · Unauthorized Access and Privilege Escalation

Warning
Location
eval-viewer/generate_review.py:287
Finding

Viewer Unconditionally Terminates Unrelated Processes Using the Requested Port

Content
View full analysis
None: """Kill any process listening on the given port.""" try: result = subprocess.run( ["lsof", "-ti", f":{port}"], capture_output=True, text=True, timeout=5, ) for pid_str in result.stdout.strip().split("\n"): if pid_str.strip(): try: os.kill(int(pid_str.strip()), signal.SIGTERM) except (ProcessLookupError, ValueError): pass if result.stdout.strip(): time.sleep(0.5) except subprocess.TimeoutExpired: pass except FileNotFoundError: print("Note: lsof not found, cannot check if port is in use", file=sys.stderr) ``` It is invoked unconditionally before binding: ```python # Kill any existing process on the target port port = args.port _kill_port(port) ``` ### Technical Analysis The viewer assumes that any process listening on the selected port is an obsolete viewer instance. It uses `lsof` to discover every listening process and sends `SIGTERM` without verifying: - Process ownership. - Executable identity. - Command-line arguments. - Whether the process was started by this application. - Whether a PID file identifies an earlier viewer instance. - Whether the user approved termination. A caller can select an arbitrary port through `--port`. Therefore, running the otherwise legitimate review command can terminate an unrelated service owned by the same operating-system user. The subprocess invocation itself does not use a shell and is not command-injectable, but the process-management policy violates least privilege. Operating-system permission checks still apply, so an unprivileged p ...[truncated 1190 chars]
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Error
Location
scripts/package_skill.py:99
Finding

Skill Packaging Follows File Symlinks and Can Archive Data Outside the Skill Directory

Content
View full analysis
/home/user/.ssh/id_rsa ``` 2. The Skill contains a valid `SKILL.md`, so `validate_skill()` succeeds. 3. The user or agent runs: ```bash python -m scripts.package_skill /path/ ...[truncated 1041 chars]
Remediation
View remediation
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
Findings (24)

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The declared description centers on skill authoring and optimization: creating new skills, modifying existing skills, improving performance, running evals, benchmarking, and optimizing descriptions for triggering. This code does not create or modify skills, run evals, optimize prompts/descriptions, or benchmark performance itself. Instead, it implements an eval-results viewer and feedback collection utility. While it is adjacent to evaluation workflows and can display benchmark.json if provided, its primary function is reviewing existing outputs and saving reviewer comments, not measuring or optimizing skill performance directly. Therefore the code's actual behavior is materially narrower and different from the declared purpose.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

There is a clear mismatch between the declared purpose and the supplied code. The description claims functionality for creating, editing, optimizing, and evaluating skills, including benchmarking and trigger-description optimization. However, the code only validates a skill folder and packages it into a .skill file using zip compression, excluding some artifacts such as pycache, node_modules, .pyc files, .DS_Store, and root-level evals. While validation is tangentially related to skill management, the primary behavior is packaging/distribution, which is not mentioned in the declared description. No code here implements creation, modification, benchmarking, variance analysis, or trigger optimization.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The declared description presents a comprehensive skill authoring, editing, optimization, and evaluation capability. In contrast, the supplied code only performs static validation of SKILL.md metadata/frontmatter in a skill directory. It does not create or modify skills, run evaluations, benchmark anything, analyze variance, or optimize descriptions. This is a materially different and much narrower primary purpose than declared, so it should be flagged as a mismatch.

Content

No source excerpt is available for this finding.

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 373)May include surrounding context.

md
1. Read the template from `assets/eval_review.html`

Agent Config Directory Access

High
Category
Agent Snooping
Confidence
85% confidence
Finding

Skill reads from agent configuration directories (.claude/, .codex/, .gemini/). These directories may contain API keys, personal settings, and other credentials that the skill has no legitimate need to access.

Content

Scanner excerpt · scripts/run_eval.py (reported line 23)May include surrounding context.

python
def find_project_root() -> Path:
    """Find the project root by walking up from cwd looking for .claude/.

    Mimics how Claude Code discovers its project root, so the command file
    we create ends up where claude -p will look for it.

Env Variable Harvesting

High
Category
Data Exfiltration
Confidence
70% confidence
Finding

Code enumerates, copies, or searches environment variables for secrets. Bulk environment access can collect credentials unrelated to the skill's stated purpose.

Content

Scanner excerpt · scripts/run_eval.py (reported line 83)May include surrounding context.

python
# Remove CLAUDECODE env var to allow nesting claude -p inside a
        # Claude Code session. The guard is for interactive terminal conflicts;
        # programmatic subprocess usage is safe.
        env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

        process = subprocess.Popen(
            cmd,

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The README instructs users to set an ANTHROPIC_API_KEY for a script that directly calls the Anthropic API, but it does not clearly warn that prompts, skill contents, or evaluation data may be transmitted to an external third-party service. In a skill-creation context, users may process proprietary prompts, internal tooling details, or sensitive benchmark data, so the omission can lead to unintended data disclosure.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
88% confidence
Finding

The skill instructs the agent to use shell, filesystem, and environment-dependent capabilities but does not declare any explicit tool scope or allowed-tools boundary. In practice this increases the chance of overbroad execution, because a caller cannot easily constrain or audit which sensitive capabilities the skill expects before invocation.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The guidance to make descriptions 'pushy' and trigger on broad adjacent contexts encourages over-triggering, which can cause the skill to be invoked in situations the user did not intend. In a skill that can read/write files, run shell commands, and launch tooling, accidental activation can lead to unnecessary access, unexpected side effects, or confusing workflow hijacking.

Content

No source excerpt is available for this finding.

Session Persistence

Medium
Category
Rogue Agent
Confidence
90% confidence
Finding

Using nohup to launch the review server creates a background process that persists beyond the immediate task, which can outlive user awareness and continue serving potentially sensitive evaluation outputs. Persistent local services increase the risk of unintended data exposure, resource leakage, and forgotten processes, especially when cleanup depends on later manual steps.

Content

Scanner excerpt · SKILL.md (reported line 247)May include surrounding context.

  1. Launch the viewer with both qualitative outputs and quantitative data:
    bash
    nohup python <skill-creator-path>/eval-viewer/generate_review.py \
      <workspace>/iteration-N \
      --skill-name "my-skill" \
      --benchmark <workspace>/iteration-N/benchmark.json \
    

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
89% confidence
Finding

Lines L189-L190 state that, when analyzing benchmark results, the analyzer's purpose is to surface patterns and anomalies rather than suggest skill improvements. However, the same agent file also instructs the analyzer earlier to generate prioritized improvement suggestions for the losing skill, creating contradictory role documentation within a single skill file about whether this analyzer suggests improvements or not.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The helper unconditionally finds any process bound to the requested port and sends it SIGTERM before starting the viewer. This can terminate unrelated local applications the user did not intend to stop, causing denial of service or data loss if the killed process was important or had unsaved state.

Content

No source excerpt is available for this finding.

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · eval-viewer/generate_review.py (reported line 291)May include surrounding context.

python
def _kill_port(port: int) -> None:
    """Kill any process listening on the given port."""
    try:
        result = subprocess.run(
            ["lsof", "-ti", f":{port}"],
            capture_output=True, text=True, timeout=5,
        )

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The main flow calls _kill_port(port) automatically before binding the server, with no warning, prompt, or dry-run behavior. In a developer tool that may be run repeatedly, this makes accidental termination of unrelated services much more likely and hides a destructive side effect behind a normal startup path.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The page auto-saves reviewer feedback to /api/feedback shortly after text entry and again on navigation, but the UI does not clearly disclose that typed content is being transmitted to a server before explicit submission. In a review tool, feedback may contain sensitive evaluation notes, prompts, or internal data, so silent background transmission creates a privacy and data-handling risk even if the endpoint is same-origin.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The script logs full prompts, model thinking, responses, and parsed descriptions to disk, and those prompts include skill content, eval queries, previous attempt history, and potentially other sensitive project data. Persisting this material without minimization, redaction, encryption, or clear operator disclosure increases the risk of secrets, proprietary prompts, user data, or chain-of-thought-like reasoning being exposed through local files, backups, CI artifacts, or shared workspaces.

Content

No source excerpt is available for this finding.

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · scripts/run_eval.py (reported line 85)May include surrounding context.

python
# programmatic subprocess usage is safe.
        env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

        process = subprocess.Popen(
            cmd,
            stdout=subprocess.PIPE,
            stderr=subprocess.DEVNULL,

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
83% confidence
Finding

The skill tells the agent to launch a browser-based review interface and potentially a local server without first requiring clear user consent. Even if the server is local, unexpected browser launches and local service startup are side effects that can surprise users and expose run artifacts on the host if bound too broadly.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
84% confidence
Finding

This markdown skill instructs the agent to save structured analysis to output_path, which is a file write affecting user or system data. The description provides no warning or disclosure that the skill will create or overwrite an output file.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
88% confidence
Finding

This markdown skill description explicitly directs the agent to write a JSON file, which affects user/system data by creating or overwriting a file. The document does not include any warning about this file-write behavior or its potential impact.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
86% confidence
Finding

This markdown skill directs the agent to save results to {outputs_dir}/../grading.json, which is a file write affecting the workspace. The description contains no explicit warning that the skill will create or overwrite a grading artifact, so users may not realize it modifies files.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
83% confidence
Finding

The HTML root sets lang="en", and the interface text throughout the page is English-only, with no indication that users may select another language or that English is a documented requirement. Under the stated policy, forcing a specific language without user opt-in can be a natural-language locale policy issue.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
84% confidence
Finding

This code writes a generated HTML report to the path supplied by the user or to an auto-created temp file, but there is no confirmation prompt or explicit user-facing disclosure at the point of write about creating or overwriting files. The same script later performs additional writes for results output, making persistent filesystem changes that are not clearly warned about in the interface beyond parameter names.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
91% confidence
Finding

Opening a browser is a side effect that affects the user's environment, and the code performs it automatically whenever reporting is enabled. Although the comment explains the intent for developers, there is no user-facing disclosure or confirmation that the script will launch a browser window/tab.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.