Back to skill

Security audit

BenchBoss

Security checks for vulnerabilities and agentic risk

Overview

The skill’s BenchBoss game-client behavior is coherent, but it asks users to run an unpinned npm MCP client that can change after review.

Install only if you trust the BenchBoss npm publisher and are comfortable running its MCP client with your local user permissions. Prefer a pinned, reviewed package version or a locked installation process, keep the local signing key private, and remember that GitHub gist registration, public bios, standings, match history, and gameplay requests are shared with the configured BenchBoss host.

Vulnerability Patterns
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T08 · Insecure Dependencies

Warning
Location
SKILL.md:17
Finding
Unpinned npm Package Is Downloaded and Executed Automatically## Vulnerability Details **File Location**: `SKILL.md`, line 17 **Vulnerability Type**: T08: Insecure Dependencies **Risk Level**: Medium **Complete Code Snippet**: ```sh npx -y @benchboss/mcp-client ``` ### Technical Analysis The Skill instructs users to run an npm package without specifying an exact version. The `npx` command resolves the package through the configured npm registry, downloads it if necessary, and executes its entry point. The `-y` option automatically accepts the installation prompt. No lockfile, package integrity hash, signature verification procedure, fixed source revision, or other reproducible dependency constraint is supplied. Consequently, the code executed at installation time can differ from the code that existed when this Skill was reviewed. Describing the package as official does not technically constrain dependency resolution or mitigate compromise of the package publisher, registry account, or release process. The external npm package was not included in the audited project and therefore its implementation could not be inspected. This finding concerns the unsafe, unpinned execution mechanism; it does not assert that the current package release is malicious. ### Attack Path 1. An attacker compromises the npm publisher account, package release pipeline, or another component capable of controlling the version resolved for `@benchboss/mcp-client`. 2. The attacker publishes a malicious release that becomes the version selected by the unpinned package reference. 3. A user follows the installation instruction in `SKILL.md`. 4. `npx -y` downloads the attacker-controlled release and executes it locally without an interactive installation confirmation. 5. The malicious package executes with the privileges of the user running the MCP host and can attempt to access user-readable files, environment variables, locally managed application material, and network resources. ### Impact Assessment Success ...[truncated 667 chars]
Remediation
## Remediation Suggestions 1. Pin the client to an exact reviewed version, for example `@benchboss/mcp-client@X.Y.Z`, rather than relying on the registry's currently resolved release. 2. Provide a lockfile with integrity metadata and use a reproducible installation process such as `npm ci`. 3. Document the verified npm publisher and canonical source repository so users can confirm package provenance. 4. Publish and verify cryptographic checksums, release signatures, or package attestations where supported. 5. Avoid `-y` for first-time installation when practical, allowing users to review the resolved package and version before execution. 6. Review package updates before changing the pinned version, including transitive dependency and lifecycle-script changes. 7. Run the MCP client with least privilege and restrict filesystem, secret, and network access to only what is required.
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
Findings (3)

Rp1

Medium
Category
MCP Rug Pull
Confidence
95% confidence
Finding
The skill instructs users to execute an unpinned npm package via `npx -y @benchboss/mcp-client`, which fetches and runs whatever version is current at execution time. This creates a supply-chain risk: if the package is compromised, replaced, or updated with unsafe behavior, the host will execute attacker-controlled code with the user's local permissions and network access.

External Transmission

Medium
Category
Data Exfiltration
Content
For a compatible local seat-token host, explicitly set BENCHBOSS_URL and BENCHBOSS_MODE=local; registration is disabled and the client carries the enqueue token. Other independent hosts supply their own authentication and connection instructions. The client reads help from the configured host; some hosts may not implement that endpoint.

- [Available games](https://api.benchboss.org/games)
- [All instruction guides](https://api.benchboss.org/instructions)
- [HTTP API reference](https://benchboss.org/docs.md)
- [For agents](https://benchboss.org/for-agents)
Confidence
50% confidence
Finding
Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

External Transmission

Medium
Category
Data Exfiltration
Content
For a compatible local seat-token host, explicitly set BENCHBOSS_URL and BENCHBOSS_MODE=local; registration is disabled and the client carries the enqueue token. Other independent hosts supply their own authentication and connection instructions. The client reads help from the configured host; some hosts may not implement that endpoint.

- [Available games](https://api.benchboss.org/games)
- [All instruction guides](https://api.benchboss.org/instructions)
- [HTTP API reference](https://benchboss.org/docs.md)
- [For agents](https://benchboss.org/for-agents)
Confidence
50% confidence
Finding
Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Static analysis

No suspicious patterns detected.