Back to skill

Security audit

Alibaba Cloud Skill Creator

Security checks for vulnerabilities and agentic risk

Overview

This skill is mostly a coherent skill-development helper, but it includes local process termination, untrusted packaging disclosure risks, and third-party model data sharing that deserve review before installation.

Use this only in trusted skill-development repositories. Review symlinks before packaging any imported skill, avoid running the eval viewer on ports used by other local services, and do not run the description-improvement tooling on confidential skill/eval data unless sending that content to Anthropic is acceptable.

Vulnerability Patterns
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (2)

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/package_skill.py:93
Finding
Skill packaging follows symbolic links and may disclose files outside the skill directory<![CDATA[ ## Vulnerability Details **File Location**: `scripts/package_skill.py`, lines 93–100 **Vulnerability Type**: Symlink-following information disclosure during archive creation **Risk Level**: Medium ### Vulnerable Code ```python for file_path in skill_path.rglob('*'): if not file_path.is_file(): continue arcname = file_path.relative_to(skill_path.parent) if should_exclude(arcname): print(f" Skipped: {arcname}") continue zipf.write(file_path, arcname) print(f" Added: {arcname}") ``` The validation performed by `scripts/quick_validate.py` does not reject symbolic links or verify that packaged files resolve inside the selected skill directory. ### Technical Analysis `Path.is_file()` follows symbolic links. Likewise, `zipfile.ZipFile.write()` opens the path and archives the contents of the file referenced by the link. The archive name is calculated from the unresolved path, so the resulting archive can make an external file appear to be an ordinary file inside the packaged skill. The exclusion logic only evaluates path names and selected directory names. It does not: - Call `is_symlink()` to reject symbolic links. - Resolve each candidate and verify containment within `skill_path`. - Use `lstat()` to distinguish regular files from links. - Restrict packaging to trusted file types or explicitly approved resources. Consequently, an untrusted imported skill can reference any file readable by the account running the packager. ### Attack Path 1. An attacker prepares a skill directory with valid `SKILL.md` frontmatter so that `validate_skill()` succeeds. 2. The attacker places a symbolic link inside the skill, for example: ```text assets/local-config.txt -> /home/victim/.config/application/credentials.json ``` 3. The victim imports the skill and runs `scripts/package_skill.py` against it. 4. `rglob('*')` discovers the symbolic link. 5. `file_path.is_file()` follows the link and returns true if its ...[truncated 929 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Reject symbolic links before adding files: ```python if file_path.is_symlink(): print(f" Skipped symlink: {file_path}") continue ``` 2. Resolve every candidate and enforce containment within the selected skill root: ```python skill_root = skill_path.resolve() for file_path in skill_path.rglob("*"): if file_path.is_symlink(): continue resolved = file_path.resolve(strict=True) if not resolved.is_relative_to(skill_root): raise ValueError(f"File escapes skill root: {file_path}") if not resolved.is_file(): continue ``` 3. Use `os.lstat()` or equivalent no-follow checks to ensure that only regular files are archived. 4. Apply containment validation to every parent component, because links in intermediate directories can also redirect traversal. 5. Fail packaging rather than silently continuing when an escaping link is detected. This makes malicious or accidental archive composition visible to the user. 6. Add automated tests covering: - A symlink to a file outside the skill directory. - A symlink to an internal file. - A symlinked directory. - Broken and cyclic links. - Nested paths containing symlinked parent directories. 7. Document that packaging untrusted skill trees is unsafe until the directory has passed link and containment validation. ]]>

T05 · Unauthorized Access and Privilege Escalation

Warning
Location
eval-viewer/generate_review.py:288
Finding
Review server startup terminates unrelated processes occupying the selected port<![CDATA[ ## Vulnerability Details **File Location**: `eval-viewer/generate_review.py`, lines 288–302 and 438–445 **Vulnerability Type**: Unauthorized local process termination **Risk Level**: Medium ### Vulnerable Code The helper identifies every process listening on the selected port and sends it `SIGTERM`: ```python def _kill_port(port: int) -> None: """Kill any process listening on the given port.""" try: result = subprocess.run( ["lsof", "-ti", f":{port}"], capture_output=True, text=True, timeout=5, ) for pid_str in result.stdout.strip().split("\n"): if pid_str.strip(): try: os.kill(int(pid_str.strip()), signal.SIGTERM) except (ProcessLookupError, ValueError): pass if result.stdout.strip(): time.sleep(0.5) except subprocess.TimeoutExpired: pass except FileNotFoundError: print("Note: lsof not found, cannot check if port is in use", file=sys.stderr) ``` The function is invoked unconditionally before the server attempts to bind: ```python # Kill any existing process on the target port port = args.port _kill_port(port) handler = partial(ReviewHandler, workspace, skill_name, feedback_path, previous, benchmark_path) try: server = HTTPServer(("127.0.0.1", port), handler) except OSError: # Port still in use after kill attempt — find a free one server = HTTPServer(("127.0.0.1", 0), handler) port = server.server_address[1] ``` ### Technical Analysis Serving a local review page does not require control over unrelated processes. The program nevertheless performs a destructive action before determining whether the requested port is available. The implementation does not verify: - Whether the listener belongs to this review tool. - Whether the process was launched by the current invocation. - Whether the user intended to stop it. - Whether terminating it could cause ...[truncated 1975 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Remove the unconditional `_kill_port(port)` call. 2. Attempt to bind the requested loopback port first. If it is unavailable, fall back to a free port: ```python handler = partial( ReviewHandler, workspace, skill_name, feedback_path, previous, benchmark_path, ) try: server = HTTPServer(("127.0.0.1", args.port), handler) except OSError: server = HTTPServer(("127.0.0.1", 0), handler) port = server.server_address[1] ``` 3. Report that the requested port was unavailable and display the selected replacement port. 4. If replacing an existing viewer is required, implement it as an explicit opt-in option such as `--replace-existing-viewer`. 5. Before terminating anything under such an option: - Confirm that the process belongs to the current user. - Verify that its executable and command line identify it as this review server. - Require interactive confirmation unless a clearly documented force flag is supplied. - Avoid signaling multiple unrelated PIDs returned for the same port. 6. Track the PID of viewers started by this tool in a user-scoped runtime directory and only stop a previously recorded, identity-verified instance. 7. Add tests verifying that: - An occupied port does not cause the existing process to be signaled. - The server falls back to a free loopback port. - The selected URL reports the actual bound port. ]]>
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
Findings (24)

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding
The declared description is about repository-maintenance help for skills: adding/importing skills, refactoring structure, improving triggers, adding smoke tests, and benchmarking skill quality before merge. The supplied code does not implement those functions. Its primary purpose is an eval-results viewer and feedback server. It discovers runs in a workspace, reads prompts/outputs/grading data, embeds text/images/PDFs/binaries into HTML, serves the page over localhost, accepts POSTed feedback, and can terminate an existing process on the selected port. Those are materially different capabilities from skill creation/migration/refactoring/testing support. No declared permissions are listed, yet the code performs filesystem reads/writes, subprocess execution, signal-based process termination, and local network serving; these are significant undeclared behaviors rather than incidental implementation details. Therefore the description does not accurately represent the code.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding
There is a material description-to-behavior mismatch. The declared purpose presents a broad repository skill-engineering assistant used for adding/importing/refactoring skills, improving triggers, adding tests, or benchmarking before merge. The actual code chunk does not implement those capabilities. Instead, it specifically loads benchmark run artifacts from disk, calculates statistics, generates benchmark summaries, and writes JSON/Markdown reports. Benchmarking is mentioned in the description, but only as one of several broad use cases; this code supports only a narrow subtask of benchmark aggregation/reporting and does not represent the larger declared behavior. No suspicious extra permissions or hidden external access are evident beyond local filesystem I/O for reading run data and writing outputs.

Tp4

High
Category
MCP Tool Poisoning
Confidence
95% confidence
Finding
The declared description presents a multi-purpose repository optimization skill covering creation, migration, imports, structural refactors, smoke tests, and benchmarking. The supplied code does not implement those broad repository-management capabilities. Its primary function is specifically to improve a skill description based on eval results using the Anthropic API. While this could be one small part of 'improving trigger descriptions,' the declared purpose is much broader and would lead a user to expect many unrelated capabilities that are absent here. The code also performs undeclared external model calls and optional transcript logging, which are concrete behaviors not reflected in the declaration. Therefore, the description does not accurately represent the actual code chunk.

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding
The declared description presents a broad repository-maintenance and skill-development utility for adding/importing/refactoring/improving/testing/benchmarking skills. The supplied code chunk instead implements a specific packaging tool: it checks that a skill directory exists, ensures SKILL.md is present, runs validate_skill, excludes certain files, and writes a .skill archive. Packaging/distribution is a materially different primary purpose from the declared creation/migration/optimization/testing functions, and this packaging capability is undeclared. The validation step is only supporting behavior, but the main behavior is clearly archive creation, so this is a description-behavior mismatch.

Tp4

High
Category
MCP Tool Poisoning
Confidence
96% confidence
Finding
The declared purpose describes a broad repository-maintenance skill used to create, migrate, import, refactor, test, and benchmark skills. The supplied code instead performs a narrow, concrete function: validating SKILL.md frontmatter and specific metadata constraints in a target skill directory. This is a materially different primary purpose. While validation could be a supporting activity in skill development, the description does not mention validation or schema checking, and none of the listed triggers align with running a CLI metadata validator.

Ae1

High
Category
analysis-evasion
Content
- Every skill must include `SKILL.md` frontmatter with `name` and `description`.
Confidence
100% confidence
Finding
Referenced artifact was not completely inspected

Ae1

High
Category
analysis-evasion
Content
- Every skill must include `SKILL.md` frontmatter with `name` and `description`.
Confidence
100% confidence
Finding
Referenced artifact was not completely inspected

Agent Config Directory Access

High
Category
Agent Snooping
Content
def find_project_root() -> Path:
    """Find the project root by walking up from cwd looking for .claude/.

    Mimics how Claude Code discovers its project root, so the command file
    we create ends up where claude -p will look for it.
Confidence
85% confidence
Finding
Skill reads from agent configuration directories (.claude/, .codex/, .gemini/). These directories may contain API keys, personal settings, and other credentials that the skill has no legitimate need to access.

Env Variable Harvesting

High
Category
Data Exfiltration
Content
# Remove CLAUDECODE env var to allow nesting claude -p inside a
        # Claude Code session. The guard is for interactive terminal conflicts;
        # programmatic subprocess usage is safe.
        env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

        process = subprocess.Popen(
            cmd,
Confidence
70% confidence
Finding
Code enumerates, copies, or searches environment variables for secrets. Bulk environment access can collect credentials unrelated to the skill's stated purpose.

Lp3

Medium
Category
MCP Least Privilege
Confidence
90% confidence
Finding
The skill describes actions that invoke local scripts, read and write repository files, and run shell commands, but it does not declare any explicit tool scope such as allowed-tools or permissions. In a repository-modifying skill, that omission weakens operator visibility and policy enforcement, increasing the chance of unintended filesystem or command execution beyond what users expect.

Natural-Language Policy Violations

Medium
Confidence
95% confidence
Finding
The instruction states that all `skills/**/SKILL.md` content must stay English-only. This is a natural-language locale restriction, and the file does not offer a user choice or explain a justified region-specific requirement for enforcing English.

Intent-Code Divergence

Medium
Confidence
95% confidence
Finding
Lines L189-L190 state that for benchmark results the analyzer's purpose is to surface patterns and anomalies, not suggest skill improvements. Earlier in the same file, the analyzer is explicitly instructed to identify loser weaknesses and generate actionable improvement suggestions, creating contradictory documentation about what this analyzer does.

Context-Inappropriate Capability

Medium
Confidence
88% confidence
Finding
The grader is instructed to verify claims against external sources, even though its primary role is to assess expectations using the transcript and produced outputs. This expands trust boundaries and can introduce nondeterminism, data exfiltration risk, or prompt-injection influence from untrusted external content, causing grading decisions to depend on information outside the evaluated run.

Context-Inappropriate Capability

Medium
Confidence
96% confidence
Finding
The viewer kills any process bound to the requested port before starting, regardless of ownership, purpose, or whether the user explicitly approved terminating it. In a local developer environment this can disrupt unrelated services, destroy active sessions, or interfere with security tooling, which exceeds the stated purpose of merely serving a review page.

subprocess module call

Medium
Category
Dangerous Code Execution
Content
def _kill_port(port: int) -> None:
    """Kill any process listening on the given port."""
    try:
        result = subprocess.run(
            ["lsof", "-ti", f":{port}"],
            capture_output=True, text=True, timeout=5,
        )
Confidence
70% confidence
Finding
subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Missing User Warnings

Medium
Confidence
94% confidence
Finding
The main flow invokes the kill helper automatically on startup, with no warning or confirmation at the point of action, so a normal launch can terminate unrelated local processes unexpectedly. In this skill context, a review-page generator has little justification for destructive system-wide side effects, making the behavior more dangerous than in an admin utility explicitly designed for port management.

Context-Inappropriate Capability

Medium
Confidence
92% confidence
Finding
The script sends full skill content, eval queries/results, and prior attempt history to Anthropic, which can expose proprietary or sensitive repository data to a third-party service. In this repository-maintenance context, external model use may be functionally intended, but the broad prompt payload is still a real data-exposure risk if skills or eval data contain secrets, internal prompts, or user-derived content.

Missing User Warnings

Medium
Confidence
94% confidence
Finding
The model call transmits skill content plus eval/query data to an external API without any disclosure or gating in this file, creating a confidentiality and privacy risk. If eval queries or skill files contain customer data, secrets, or internal instructions, they may be unnecessarily exposed outside the local environment.

Missing User Warnings

Medium
Confidence
80% confidence
Finding
The CLI advertises `--verbose` as 'Print thinking to stderr', and the function captures detailed model thinking derived from prompts containing skill content, history, and eval queries. This creates a disclosure risk because sensitive or proprietary material may be echoed to terminal logs without any warning in the file.

subprocess module call

Medium
Category
Dangerous Code Execution
Content
# programmatic subprocess usage is safe.
        env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

        process = subprocess.Popen(
            cmd,
            stdout=subprocess.PIPE,
            stderr=subprocess.DEVNULL,
Confidence
70% confidence
Finding
subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Missing User Warnings

Low
Confidence
81% confidence
Finding
This markdown skill instructs the agent to save structured analysis to `output_path`, which is a file-writing operation that can affect user data or the filesystem. The document does not include any warning, confirmation, or disclosure about overwriting or creating files when performing this action.

Missing User Warnings

Low
Confidence
79% confidence
Finding
The benchmark-analysis section tells the agent to save notes to `output_path` as a JSON array, which is another file write affecting the user's filesystem. The markdown does not warn the user that this skill writes output files or clarify overwrite behavior.

Missing User Warnings

Low
Confidence
91% confidence
Finding
The skill instructs the agent to write a JSON results file to a specified path or a default filename, but it does not require confirming the destination, checking for existing files, or warning about overwriting local data. In an agent setting where paths may come from upstream inputs, this can lead to unintended file modification or overwrite of user workspace artifacts.

Intent-Code Divergence

Low
Confidence
78% confidence
Finding
The top-level documentation frames the behavior narrowly as using eval results, while the prompt construction additionally includes complete skill content and prior optimization history before invoking the model. This is not just omitted detail in implementation flow; it changes the actual inputs the script relies on when producing output.

Static analysis

No suspicious patterns detected.