Back to skill

Security audit

fusion-bench

Security checks for vulnerabilities and agentic risk

Overview

This is a straightforward FusionBench guide; the main caution is that it tells users to install an unpinned Python package and run model benchmark commands.

Install FusionBench in a virtual environment or container, avoid running package installs with administrator privileges, and pin or verify package versions if reproducibility or supply-chain control matters. Expect benchmark runs to download or cache model artifacts and use meaningful compute resources.

Vulnerability Patterns
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T08 · Insecure Dependencies

Warning
Location
SKILL.md:17
Finding

Unpinned Third-Party Package Installation

Content
View full analysis

Vulnerability Details

File Location: SKILL.md, lines 17-19; also referenced at line 12
Vulnerability Type: Unpinned dependency installation from a mutable package registry
Risk Level: Medium

Vulnerable Code

bash
# Install
pip install fusion-bench

The same unpinned installation command is advertised at line 12:

markdown
**PyPI:** `pip install fusion-bench`

Technical Analysis

The Skill directs users or agents to install fusion-bench from PyPI without specifying a reviewed version, verifying package hashes, using a lockfile, or requiring an isolated environment. Consequently, the package and its transitive dependencies are resolved from mutable registry state at installation time.

Python package installation can execute build-system and installation logic. If a future release of the named package, one of its dependencies, or the associated publishing account is compromised, following this instruction could execute attacker-controlled code with the privileges of the user running pip. The audit found no evidence that the currently published package is malicious; the vulnerability is the absence of controls ensuring that the installed artifact is the version that was reviewed and intended.

Attack Path

  1. An attacker compromises the package publisher, a required transitive dependency, or the relevant supply-chain publishing process.
  2. The attacker publishes a malicious version that satisfies the unrestricted dependency resolution request.
  3. A user or agent follows the Skill and runs pip install fusion-bench.
  4. pip resolves the attacker-controlled release because no version or artifact hash is pinned.
  5. Malicious build, installation, import-time, or runtime code executes in the installation environment.

Impact Assessment

Successful exploitation could provide arbitrary code execution with the permissions of the account that invokes pip. Depending on that ...[truncated 350 chars]

Remediation
View remediation

Remediation Suggestions

  • Pin fusion-bench to a specific version that has been reviewed, for example fusion-bench==X.Y.Z.
  • Publish a lockfile that pins all transitive dependencies.
  • Provide cryptographic hashes and install with pip install --require-hashes -r requirements.txt.
  • Recommend installation inside a dedicated virtual environment or disposable container rather than a system Python environment.
  • Avoid running package installation with administrator or root privileges.
  • Document the expected official package publisher and source repository so users can validate provenance.
  • Establish a dependency-update process that reviews and tests new versions before changing the pinned version or hashes.
Vulnerability Patterns
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Findings (2)

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The skill description is broad enough to trigger on generic requests about model merging, evaluation, or working with model pools, which can cause the agent to invoke this skill outside a narrowly intended FusionBench context. Over-broad activation increases the chance of inappropriate tool use, unexpected package installation, external repo reliance, or execution of benchmark commands when the user's request was only informational.

Content

No source excerpt is available for this finding.

Session Persistence

Medium
Category
Rogue Agent
Confidence
60% confidence
Finding

Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.

Content

Scanner excerpt · SKILL.md (reported line 149)May include surrounding context.

Adding a New Method

Step 1: Create method file

python
# fusion_bench/method/my_method.py