ClawHub
Back to skill

Security audit

Skill Evaluator

Security checks for vulnerabilities and agentic risk

Overview

This skill is a coherent skill-evaluation helper; its main risk is unpinned optional/required third-party tooling, not hidden or malicious behavior.

Before installing or using it, treat the bundled evaluator as a local file-reading checker over the skill directory you point it at. Prefer pinning PyYAML and any optional SkillLens version, and run third-party scanners with only the directories and credentials needed for the audit.

A.I.G

Vulnerability Patterns
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T08 · Insecure Dependencies

Warning
Location
SKILL.md:65
Finding

Unpinned Third-Party Package Retrieval and Execution

Content
View full analysis

Vulnerability Details

File Location: SKILL.md:65 and SKILL.md:69; related dependency import at scripts/eval-skill.py:20
Vulnerability Type: Unpinned third-party dependencies and mutable package execution
Risk Level: Medium

Vulnerable Code

markdown
This evaluator covers security basics (credentials, input validation, data safety) but for thorough security audits of skills under development, consider [SkillLens](https://www.npmjs.com/package/skilllens) (`npx skilllens scan <path>`).
markdown
- PyYAML (`pip install pyyaml`) — for frontmatter parsing in automated checks

The installed PyYAML package is subsequently loaded by the evaluator:

python
import yaml

Technical Analysis

The documented npx skilllens scan &lt;path&gt; command may retrieve and immediately execute whichever package version the npm registry resolves at invocation time. The instruction does not pin a reviewed version, require a lockfile, verify an integrity hash, or require installation and inspection before execution. Consequently, the effective code can change after this Skill has been audited.

The pip install pyyaml instruction similarly installs a mutable, unpinned package version. The evaluator imports that package at startup, so package initialization code executes with the evaluator's privileges. The evaluator itself appropriately uses yaml.safe_load; the issue is dependency provenance and version mutability rather than unsafe YAML deserialization.

Attack Path

  1. An attacker compromises a relevant registry account, package release, maintainer environment, or transitive dependency.
  2. The attacker publishes malicious package content under a version that an unpinned command can resolve.
  3. A user follows SKILL.md and runs npx skilllens scan &lt;path&gt; or installs unpinned PyYAML.
  4. The package registry supplies mutable third-party code without local integrity verification.
  5. The downloaded code executes under the inv ...[truncated 793 chars]
Remediation
View remediation

Remediation Suggestions

  1. Pin SkillLens to an exact reviewed version:

    bash
    npx --yes skilllens@REVIEWED_VERSION scan /path/to/skill

  2. Prefer installing dependencies through a committed lockfile and then using npx --no-install so the scan cannot silently retrieve a different release:

    bash
    npm ci
npx --no-install skilllens scan /path/to/skill

  3. Verify npm lockfile integrity and package provenance before execution. Run external scanners in a restricted environment with minimal filesystem access, no unnecessary credentials, and controlled network access.

  4. Pin PyYAML to a reviewed version or tightly bounded compatible range. For reproducible installation, use a requirements file with cryptographic hashes:

    text
    PyYAML==REVIEWED_VERSION --hash=sha256:VERIFIED_HASH

    Install it with:

    bash
    python3 -m pip install --require-hashes -r requirements.txt

  5. Clearly label SkillLens as an optional external tool that downloads and executes third-party code, and advise users not to expose unrelated sensitive directories or credentials to the scanning process.

SkillSpector

Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
Findings (7)

Anti-Refusal Statement

High
Category
Anti-Refusal
Confidence
80% confidence
Finding

Skill instructs the agent to omit warnings, disclaimers, or ethical commentary. Stripping safety caveats hides risk from the user and is a common jailbreak preamble.

Content

Scanner excerpt · references/rubric.md (reported line 147)May include surrounding context.

md
- **3:** Confirmation on destructive ops. Some recoverability.
- **2:** Confirmation exists but permanent is the default.
- **1:** No confirmation on destructive operations.
- **0:** Destructive operations with no warning.

---

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
82% confidence
Finding

The skill advertises commands that invoke local scripts and an external scanner, but it does not declare any explicit tool scope such as allowed-tools or permissions. That creates an authorization ambiguity: a caller or platform may permit broader capabilities than necessary, increasing the chance of unintended shell, network, file, or environment access during evaluation workflows.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The description says to use the skill when asked to "review, audit, evaluate, score, or assess a skill," which is a broad set of common verbs without clear exclusion conditions. Although scoped to skills, it does not provide explicit trigger boundaries or negative examples, which can lead to unintended invocation in general conversations about reviewing or assessing.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
89% confidence
Finding

The documentation recommends running npx skilllens without pinning an exact version. This is a supply-chain risk because npx will resolve and execute the latest published package by default, allowing unexpected behavior changes or a compromised upstream release to run code in the evaluator's environment.

Content

No source excerpt is available for this finding.

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
75% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · references/rubric.md (reported line 146)May include surrounding context.

md
- **4:** Destructive ops default to recoverable (trash). Undo available. Confirmation prompts.
- **3:** Confirmation on destructive ops. Some recoverability.
- **2:** Confirmation exists but permanent is the default.
- **1:** No confirmation on destructive operations.
- **0:** Destructive operations with no warning.

---

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
75% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · references/rubric.md (reported line 174)May include surrounding context.

md
- **4:** Dry-run defaults, confirmation prompts, safe defaults (trash > delete).
- **3:** Most write ops have confirmation. Safe defaults.
- **2:** Some write ops unprotected.
- **1:** Write ops fire without confirmation.
- **0:** Silent data destruction possible.

---

Session Persistence

Medium
Category
Rogue Agent
Confidence
75% confidence
Finding

Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.

Content

Scanner excerpt · scripts/eval-skill.py (reported line 315)May include surrounding context.

python
"html", "http", "imaplib", "importlib", "inspect", "io", "ipaddress",
        "itertools", "json", "keyword", "linecache", "locale", "logging",
        "lzma", "math", "mimetypes", "multiprocessing", "operator", "os",
        "pathlib", "pickle", "platform", "plistlib", "pprint", "profile",
        "queue", "random", "re", "readline", "reprlib", "secrets",
        "select", "shelve", "shlex", "shutil", "signal", "smtplib",
        "socket", "sqlite3", "ssl", "stat", "statistics", "string",

Static analysis

No suspicious patterns detected.