Back to skill

Security audit

skill-creator

Security checks for vulnerabilities and agentic risk

Overview

This skill mostly matches its stated purpose, but its local review viewer and packager have real flaws that could expose, corrupt, or unexpectedly affect local data when used on untrusted skills.

Install only if you are comfortable with a skill-authoring tool that edits files, runs eval subprocesses, launches a local review server, and packages skill archives. Avoid using the viewer or packager on untrusted skill outputs/directories until the inline-script embedding, localhost feedback authorization, automatic port-kill behavior, and symlink packaging issues are fixed.

Vulnerability Patterns
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (4)

T09 · Insecure Skill Coding Practices

Error
Location
eval-viewer/generate_review.py:271
Finding

Stored Script Injection Through Embedded Evaluation Output

Content
View full analysis
// ---- Embedded data (injected by generate_review.py) ---- /*__EMBEDDED_DATA__*/ ``` ### Technical Analysis Evaluation prompts, generated output files, grading evidence, and previous feedback are collected into `embedded` and serialized with `json.dumps()`. The resulting JSON is then inserted directly into an inline JavaScript block. JSON serialization alone does not make untrusted data safe for embedding in an HTML ``. A malicious or compromised Skill can create an output file containing content similar to: ```html ``` When the review HTML is generated, this content remains inside the serialized data. The browser's HTML parser terminates the original script at `` before JavaScript parsing occurs, causing the injected script element to execute. This behavior is especially dangerous because model-generated evaluation artifacts are not necessarily trustworthy. The viewer is explicitly designed to open and render those artifacts, creating a direct stored-injection path. ### Attack Path 1. A user evaluates an untrusted or compromised Skill. 2. The Skill produces a text output containing a closing `` sequence followed by attacker-controlled JavaScript. 3. `embed_file()` reads the output and stores it as the file's `content ...[truncated 1233 chars]
Remediation
View remediation
... ``` 3. Escape characters that are significant to the HTML parser before insertion, including `<`, `>`, `&`, U+2028, and U+2029. For example, replace `<` with `\u003c` in the serialized representation. 4. Read the data using `textContent` and parse it explicitly: ```javascript const EMBEDDED_DATA = JSON.parse( document.getElementById("embedded-data").textContent ); ``` 5. Add a restrictive Content Security Policy. At minimum, restrict `connect-src` to the local viewer and prevent arbitrary script execution. 6. Add regression tests using output strings containing ``, HTML tags, Unicode separators, and nested JSON content. 7. Treat all evaluation artifacts as hostile input, even when they were produced by an AI agent rather than directly supplied by a user. ]]>

T09 · Insecure Skill Coding Practices

Warning
Location
eval-viewer/generate_review.py:355
Finding

Local Feedback Endpoint Permits Unauthenticated Cross-Origin State Changes

Content
View full analysis
None: if self.path == "/api/feedback": length = int(self.headers.get("Content-Length", 0)) body = self.rfile.read(length) try: data = json.loads(body) if not isinstance(data, dict) or "reviews" not in data: raise ValueError("Expected JSON object with 'reviews' key") self.feedback_path.write_text(json.dumps(data, indent=2) + "\n") resp = b'{"ok":true}' self.send_response(200) except (json.JSONDecodeError, OSError, ValueError) as e: resp = json.dumps({"error": str(e)}).encode() self.send_response(500) self.send_header("Content-Type", "application/json") self.send_header("Content-Length", str(len(resp))) self.end_headers() self.wfile.write(resp) else: self.send_error(404) ``` ### Technical Analysis The local HTTP server accepts a write request to `/api/feedback` without authenticating the requester or validating the request's `Origin`, `Host`, content type, or a per-session anti-CSRF token. The endpoint parses the body as JSON regardless of the declared content type. Consequently, a malicious web page may attempt to send a cross-origin request using a content type that avoids a CORS preflight, such as `text/plain`, while placing valid JSON in the body. Browser protections may prevent the attacking page from reading the response, but they do not universally prevent the request from reaching a localhost service. Modern browser private-network restrictions may mitigate some variants, but they should not be relied upon as endpoint authorization. The server then overwrites `feedback.json` with the supplied object. It valida ...[truncated 1444 chars]
Remediation
View remediation

T05 · Unauthorized Access and Privilege Escalation

Warning
Location
eval-viewer/generate_review.py:287
Finding

Viewer Terminates Unrelated Processes Occupying Its Configured Port

Content
View full analysis
None: """Kill any process listening on the given port.""" try: result = subprocess.run( ["lsof", "-ti", f":{port}"], capture_output=True, text=True, timeout=5, ) for pid_str in result.stdout.strip().split("\n"): if pid_str.strip(): try: os.kill(int(pid_str.strip()), signal.SIGTERM) except (ProcessLookupError, ValueError): pass if result.stdout.strip(): time.sleep(0.5) except subprocess.TimeoutExpired: pass except FileNotFoundError: print("Note: lsof not found, cannot check if port is in use", file=sys.stderr) ``` The function is called unconditionally before the server binds: ```python # Kill any existing process on the target port port = args.port _kill_port(port) handler = partial( ReviewHandler, workspace, skill_name, feedback_path, previous, benchmark_path, ) try: server = HTTPServer(("127.0.0.1", port), handler) except OSError: server = HTTPServer(("127.0.0.1", 0), handler) port = server.server_address[1] ``` ### Technical Analysis The viewer uses `lsof` to identify every process listening on the requested port and sends `SIGTERM` to each resulting PID. It does not verify that the process is: - A previous instance of this viewer. - Owned by the expected application. - Associated with the current workspace. - Safe to terminate. - Stale or malfunctioning. The subsequent code already supports binding to an automatically selected free port after a bind failure. Therefore, terminating another process is unnecessary for the viewer's d ...[truncated 1346 chars]
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/package_skill.py:94
Finding

Skill Packager Follows File Symlinks Outside the Skill Directory

Content
View full analysis
/home/user/.config/application/credentials.json ``` When the user packages that Skill, the contents of the external target can be included in the generated `.skill` archive under an innocuous archive name. ### Attack Path 1. An attacker supplies or contributes a Skill directory containing a symbolic link to a predictable sensitive local file. 2. The user runs `scripts/package_skill.py` against that directory. 3. `rglob()` discovers the symbolic-link entry. 4. `is_file()` follows the link and treats it as a valid file. 5. `zipf.write()` reads the external target and adds its contents to the archive. 6. The generated `.skill` file is shared, uploaded, or returned to the attacker. 7. The attacker extracts the a ...[truncated 768 chars]
Remediation
View remediation
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
Findings (18)

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The skill also packages directories into distributable .skill archives and performs validation/exclusion logic, but this behavior is not reflected in the declared purpose. Packaging changes distribution semantics and can cause unintended disclosure or propagation of bundled content if a user did not expect artifact creation. While not inherently malicious, undisclosed packaging functionality broadens the operational scope of the skill.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The skill also packages directories into distributable .skill archives and performs validation/exclusion logic, but this behavior is not reflected in the declared purpose. Packaging changes distribution semantics and can cause unintended disclosure or propagation of bundled content if a user did not expect artifact creation. While not inherently malicious, undisclosed packaging functionality broadens the operational scope of the skill.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding

The skill also packages directories into distributable .skill archives and performs validation/exclusion logic, but this behavior is not reflected in the declared purpose. Packaging changes distribution semantics and can cause unintended disclosure or propagation of bundled content if a user did not expect artifact creation. While not inherently malicious, undisclosed packaging functionality broadens the operational scope of the skill.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
96% confidence
Finding

The guidance explicitly recommends making skill descriptions 'pushy' and triggering on broad related contexts even when the user does not explicitly request the capability. For a skill with shell, file, packaging, browser, and background-process behaviors, overbroad triggering materially increases the chance of accidental activation in unrelated workflows. That can lead to unnecessary command execution, file modifications, or data exposure in contexts where the user did not intend to invoke this complex skill.

Content

No source excerpt is available for this finding.

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 364)May include surrounding context.

md
1. Read the template from `assets/eval_review.html`

Agent Config Directory Access

High
Category
Agent Snooping
Confidence
85% confidence
Finding

Skill reads from agent configuration directories (.claude/, .codex/, .gemini/). These directories may contain API keys, personal settings, and other credentials that the skill has no legitimate need to access.

Content

Scanner excerpt · scripts/run_eval.py (reported line 23)May include surrounding context.

python
def find_project_root() -> Path:
    """Find the project root by walking up from cwd looking for .claude/.

    Mimics how Claude Code discovers its project root, so the command file
    we create ends up where claude -p will look for it.

Env Variable Harvesting

High
Category
Data Exfiltration
Confidence
70% confidence
Finding

Code enumerates, copies, or searches environment variables for secrets. Bulk environment access can collect credentials unrelated to the skill's stated purpose.

Content

Scanner excerpt · scripts/run_eval.py (reported line 83)May include surrounding context.

python
# Remove CLAUDECODE env var to allow nesting claude -p inside a
        # Claude Code session. The guard is for interactive terminal conflicts;
        # programmatic subprocess usage is safe.
        env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

        process = subprocess.Popen(
            cmd,

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
80% confidence
Finding

The skill instructs the agent to use shell, file read/write, environment access, and process control, but it does not declare any tool or permission scope in metadata. That creates an unclear trust boundary: a user invoking a seemingly ordinary skill-authoring tool may not realize it can execute commands, modify files, and launch background processes. In a skill ecosystem, undeclared capabilities increase the chance of over-privileged execution and unsafe reuse.

Content

No source excerpt is available for this finding.

Session Persistence

Medium
Category
Rogue Agent
Confidence
72% confidence
Finding

The skill instructs use of nohup to launch a background viewer process and later kill it, which creates session persistence outside the immediate interaction. Background services can outlive the user's awareness, continue serving local files, or accumulate stale processes if cleanup fails. In this skill's context, persistence is more concerning because the viewer may expose workspace outputs and feedback artifacts through a local HTTP endpoint.

Content

Scanner excerpt · SKILL.md (reported line 238)May include surrounding context.

  1. Launch the viewer with both qualitative outputs and quantitative data:
    bash
    nohup python <skill-creator-path>/eval-viewer/generate_review.py \
      <workspace>/iteration-N \
      --skill-name "my-skill" \
      --benchmark <workspace>/iteration-N/benchmark.json \
    

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · eval-viewer/generate_review.py (reported line 291)May include surrounding context.

python
def _kill_port(port: int) -> None:
    """Kill any process listening on the given port."""
    try:
        result = subprocess.run(
            ["lsof", "-ti", f":{port}"],
            capture_output=True, text=True, timeout=5,
        )

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The script automatically kills whatever process is bound to the requested port before starting its local server, with no confirmation or ownership check. In a developer workstation or shared environment, this can terminate unrelated services, interrupt security tools, or destroy active local workflows, making it a potentially unsafe side effect for a review utility.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

This HTML/JavaScript file performs a network POST to /api/feedback containing review text and timestamps, which is user-provided data. Although the save is implicit and the UI later shows "Saved," there is no advance warning, confirmation, or clear disclosure that feedback is transmitted to a server during typing.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The script writes a transcript containing the full prompt, model thinking, response text, skill content, eval results, and previous attempt history to disk. Those fields can contain proprietary skill logic, user-like test queries, and sensitive chain-of-thought/reasoning that should not be persisted by default; if the log directory is accessible to other users, backup systems, or telemetry pipelines, this becomes an unintended data exposure risk.

Content

No source excerpt is available for this finding.

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · scripts/run_eval.py (reported line 85)May include surrounding context.

python
# programmatic subprocess usage is safe.
        env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

        process = subprocess.Popen(
            cmd,
            stdout=subprocess.PIPE,
            stderr=subprocess.DEVNULL,

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
94% confidence
Finding

The skill instructs the agent to write a JSON results file to a specified path (or a default file) without any guardrails about validating the destination, warning about overwrites, or limiting writes to a safe workspace. In an agent context, this can lead to unintended file modification or overwriting of existing data if the provided path is sensitive, incorrect, or attacker-controlled.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
81% confidence
Finding

The code issues a network request to /api/feedback and loads prior review content into the interface, but the user-facing instructions do not explain that saved server-side feedback may be fetched automatically. This is a data-handling behavior relevant to privacy and should be disclosed.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
80% confidence
Finding

This code writes benchmark.json and benchmark.md to disk, potentially overwriting existing files at the computed output paths. Although the script prints messages after writing, there is no pre-write confirmation or explicit warning comment/docstring near the write operation that alerts users about the overwrite behavior.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
82% confidence
Finding

This code launches an external subprocess to run claude -p, which is a safety-relevant operation under the warning criteria for code files. Although the function docstring explains the behavior to developers, there is no user-facing prompt, confirmation, or runtime disclosure before execution in this script.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.