SkillBench

• Outcome: Review
• Latest audit: Sep 10, 2026 · 4:27 PM UTC
• Version: 2.0.0

Overview

The skill is a coherent benchmark helper, but it installs an unpinned global CLI and documents automatic ClawVault syncing without enough data-control detail.

Install only if you are comfortable with a globally installed npm CLI and potential ClawVault sync of benchmark metadata. Prefer a pinned package version, inspect the CLI source/package provenance, and confirm sync can be disabled or reviewed before recording sensitive task names or errors.

A.I.G

Vulnerability Patterns

• Insecure Dependencies: Introduces malicious components through unsafe dependency sources
• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review

Findings (2)

T08 · Insecure Dependencies

Warning

• Location: SKILL.md:7
• Finding: Unpinned Global npm Dependency Prevents Verification of Executable Behavior## Vulnerability Details **File Location**: `SKILL.md`, lines 7-12 and 24-28 **Vulnerability Type**: Unpinned third-party dependency installed globally **Risk Level**: Medium ### Complete Code Snippet ```yaml requires: bins: [skillbench] install: - id: node kind: node package: "@versatly/skillbench" bins: [skillbench] label: Install SkillBench CLI (npm) ``` ```markdown ## Installation ```bash npm install -g @versatly/skillbench ``` ``` ### Technical Analysis The skill delegates all executable behavior to the external npm package `@versatly/skillbench`. The package is installed globally without an exact version, integrity hash, lockfile, or included source code. Because npm resolves the current package release at installation time, the code executed by users can differ from the code available when this skill was reviewed. Global installation also exposes the package's installation lifecycle scripts and executable code to the permissions of the invoking user. The supplied project contains no implementation with which to verify the package's behavior. This is a supply-chain weakness rather than evidence that the currently published package is malicious. ### Attack Path 1. An attacker compromises the npm publisher account, package repository, or release process for `@versatly/skillbench`. 2. The attacker publishes a malicious release under the same package name. 3. A user follows the documented `npm install -g @versatly/skillbench` instruction without an exact version or integrity constraint. 4. npm resolves and installs the attacker-controlled release. 5. Package lifecycle scripts or the installed CLI execute with the invoking user's privileges. 6. The malicious package can access resources available to that user, subject to operating-system controls. ### Impact Assessment Successful exploitation could permit arbitrary code execution with the privileges of the user performing the installation or invoking the CLI. Potentially exposed ...[truncated 288 chars]
• Remediation: ## Remediation Suggestions - Pin the dependency to a reviewed exact version, such as `@versatly/skillbench@X.Y.Z`. - Publish and verify the expected npm integrity digest and package provenance. - Include a lockfile or equivalent immutable dependency manifest where the installation model permits it. - Avoid global installation; use a project-local dependency or a version-pinned `npx` invocation with appropriate safeguards. - Audit package lifecycle scripts and the CLI source before recommending installation. - Link to the authoritative source repository and reproducible build information. - Use automated dependency monitoring and require review before updating the pinned version. - Document that installation must occur under a non-privileged account.

T09 · Insecure Skill Coding Practices

Warning

• Location: SKILL.md:144
• Finding: Automatic External Synchronization Is Documented Without Clear Data-Control Boundaries## Vulnerability Details **File Location**: `SKILL.md`, lines 83-88 and 144-149 **Vulnerability Type**: Insufficiently documented external data synchronization **Risk Level**: Medium ### Complete Code Snippet ```markdown ### Sync ```bash skillbench sync --clawhub # Import installed skills skillbench sync --vault # Sync to ClawVault skillbench sync --all # Everything ``` ``` ```markdown ## ClawVault Integration Benchmarks sync to [ClawVault](https://clawvault.dev) automatically. ``` ### Technical Analysis The document states that benchmark information synchronizes automatically with the external ClawVault service. Elsewhere, the documented benchmark records include task names, success or failure status, duration, and error types. Such metadata may reveal project activity, workflow characteristics, operational failures, or internal task descriptions. The documentation does not define: - Whether automatic synchronization is enabled by default. - The exact fields included in synchronization. - The destination API endpoints. - When user consent is obtained. - How authentication credentials are stored. - Whether synchronization can be disabled. - Data retention, deletion, and redaction behavior. The supplied project contains only `SKILL.md`, so the implementation and actual network behavior cannot be verified. The finding is therefore limited to the unsafe or ambiguous behavior explicitly documented; it does not establish that credentials or file contents are exfiltrated. ### Attack Path 1. A user installs and uses the SkillBench CLI. 2. The user records benchmarks containing task names, durations, outcomes, or error classifications. 3. According to the documentation, benchmark information synchronizes automatically with ClawVault. 4. If synchronization occurs without an explicit confirmation boundary or adequate redaction, task metadata leaves the local environment. 5. Anyone with authorized or compromised ac ...[truncated 633 chars]
• Remediation: ## Remediation Suggestions - Make remote synchronization explicitly opt-in and disabled by default. - Require informed confirmation before the first transmission. - Display the destination endpoint and exact data fields before synchronization. - Provide a documented local-only mode. - Allow synchronization to be disabled permanently through configuration. - Redact or reject secrets and sensitive text in task names and error details. - Minimize transmitted fields and use pseudonymous identifiers where possible. - Encrypt all synchronization traffic using current TLS configurations. - Store authentication tokens using an operating-system credential store rather than plaintext configuration. - Document retention, deletion, access control, and incident-response policies. - Add verbose or dry-run modes that show the proposed payload without transmitting it. - Include implementation source and tests demonstrating that unrelated files, environment variables, and credentials are never included.

SkillSpector

Vulnerability Patterns

• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Rogue Agent: Self-Modification, Session Persistence
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access
• Supply Chain: Unpinned Dependencies, External Script Fetching, Obfuscated Code

Findings (2)

Self-Modification

High

• Category: Rogue Agent

Content

2. Do the task    → tt start "Create PR" && ... && tt stop
3. Record result  → skillbench record "Create PR" --success
4. Check scores   → skillbench score github
5. Improve skill  → Update skill, bump version
6. Repeat         → Compare v1.0.0 vs v1.1.0
```

• Confidence: 85% confidence
• Finding: Skill modifies its own code, configuration, or behavior at runtime. Self-modification enables an agent to escalate privileges, disable safety constraints, or install persistent backdoors.

Missing User Warnings

Medium

• Confidence: 93% confidence
• Finding: The skill states that benchmarks sync to ClawVault automatically but does not disclose what data is transmitted, when transmission occurs, or how users can opt out. Benchmark records may include task names, timing, failures, and other operational metadata that could expose sensitive workflow details if sent off-host without explicit consent.

Static analysis

No suspicious patterns detected.