Back to skill

Security audit

Test Runner

Security checks for vulnerabilities and agentic risk

Overview

This skill is a straightforward testing guide; its package-install commands carry normal supply-chain caution but are disclosed and aligned with running tests.

Before installing, prefer the project's existing lockfile, package scripts, and locally installed test tools. Ask the agent to get approval before adding dependencies or Playwright browsers, and pin or review versions for sensitive projects.

Vulnerability Patterns
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T08 · Insecure Dependencies

Warning
Location
SKILL.md:29
Finding

Unpinned Third-Party Dependency Installation and Execution

Content
View full analysis

Vulnerability Details

File Location: SKILL.md, lines 29–78
Vulnerability Type: Uncontrolled installation and execution of mutable third-party dependencies
Risk Level: Medium

Vulnerable Code

bash
npm install -D vitest @testing-library/react @testing-library/jest-dom
npx vitest
npx vitest run
npx vitest --coverage
bash
npm install -D jest @types/jest ts-jest
npx jest
npx jest --watch
npx jest --coverage
npx jest path/to/test
bash
uv pip install pytest pytest-cov pytest-asyncio httpx
pytest
pytest -v
pytest -x
pytest --cov=app
pytest tests/test_api.py -k "test_login"
pytest --tb=short
bash
npm install -D @playwright/test
npx playwright install
npx playwright test
npx playwright test --headed
npx playwright test --debug
npx playwright test --project=chromium
npx playwright show-report

Technical Analysis

The skill instructs an agent to install packages without exact version pins, lockfile enforcement, package-integrity validation, registry restrictions, or explicit user approval. Package names consequently resolve to mutable releases available from the configured package registry at execution time.

Package installation can execute lifecycle scripts with the permissions of the agent process. In addition, npx may download and execute a package dynamically if a trusted local executable is unavailable. The npx playwright install command also retrieves browser executables after package installation. These behaviors create a supply-chain execution boundary that is not controlled by the reviewed skill content.

There is no evidence that the named packages are currently malicious or that the skill intentionally selects a typosquatted package. The vulnerability is the unsafe dependency acquisition and execution procedure, which could expose users to a compromised package release, registry substitution, dependency confusion in a manipul ...[truncated 1837 chars]

Remediation
View remediation

Remediation Suggestions

  1. Require explicit user approval before installing any new package, browser binary, or other executable component.
  2. Prefer the project's existing test framework, lockfile, package scripts, and locally installed executables.
  3. Pin dependencies to exact, reviewed versions rather than unconstrained package names.
  4. For npm projects, use a committed lockfile and npm ci rather than generating new dependency resolutions with npm install.
  5. Invoke tools through verified project scripts or explicitly local binaries, and prevent npx from downloading missing packages, for example by using npx --no-install where supported.
  6. For Python projects, install from a reviewed requirements or lock file containing exact versions and cryptographic hashes.
  7. Restrict package managers to approved registries and verify that project-level configuration cannot silently redirect dependency resolution.
  8. Disable package lifecycle scripts where compatible with the selected framework, then explicitly run only reviewed setup operations.
  9. Run installation and tests in an isolated environment with minimal filesystem permissions, no unnecessary credentials, and restricted outbound network access.
  10. Treat Playwright browser downloads as executable dependency acquisition: pin the Playwright version, verify download provenance and integrity, and obtain approval before installation.
Vulnerability Patterns
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
Findings (15)

Rp1

Medium
Category
MCP Rug Pull
Confidence
91% confidence
Finding

The skill instructs use of npx vitest without pinning a version, which can cause the agent to fetch and execute whatever version is current at runtime. In an agent skill that may run commands automatically, this creates a supply-chain risk because a compromised or malicious newly published package version could be executed unexpectedly.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
91% confidence
Finding

The unpinned npx vitest run invocation may download and execute the latest package version instead of a reviewed, fixed version. That behavior is risky in automation contexts because command execution can vary over time and expose the environment to malicious upstream changes.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
91% confidence
Finding

Using npx vitest --coverage without a fixed version permits runtime resolution of an unreviewed package version. In a skill designed to run tests, that means the model could execute third-party code from the registry with integrity and reproducibility risks.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
90% confidence
Finding

The npx jest command is unpinned, so it may resolve to whatever version is current when the skill is used. That introduces supply-chain and reproducibility risk because an agent may execute unexpected code from the npm registry.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
90% confidence
Finding

npx jest --watch inherits the same unpinned package execution risk as other npx examples. In an agent skill, the risk is somewhat elevated because the instructions are prescriptive and likely to be followed automatically.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
90% confidence
Finding

The skill recommends npx jest --coverage without constraining the resolved package version. This can result in execution of untrusted or changed upstream code, making builds non-reproducible and potentially unsafe.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
90% confidence
Finding

npx jest path/to/test is another unpinned runtime package execution pattern. Because the skill is about test execution, these examples are central to expected behavior and therefore meaningfully increase the chance that unsafe package resolution will happen in practice.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
92% confidence
Finding

The npx playwright install instruction is unpinned and may fetch an arbitrary current Playwright version at runtime. This is a supply-chain concern, especially because Playwright installation can also download browser binaries, increasing the amount of externally sourced code and artifacts brought into the environment.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
92% confidence
Finding

npx playwright test allows execution of whatever Playwright version is resolved at the time of use. In an automated agent setting, this weakens integrity guarantees and can expose the host to malicious or compromised upstream packages.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
92% confidence
Finding

The unpinned npx playwright test --headed command poses the same package-resolution risk as other npx commands. Because Playwright often interacts with browser processes and system resources, executing an unexpected version can have broader operational impact.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
92% confidence
Finding

npx playwright test --debug is unpinned and can execute unreviewed upstream code. The skill context makes this more dangerous than a passive document because the content is directly instructing execution of commands in developer or agent environments.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
92% confidence
Finding

The command npx playwright test --project=chromium still relies on unpinned package resolution. This creates a repeatable path for running code from the registry without version control, undermining trust and reproducibility.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
92% confidence
Finding

npx playwright show-report is another instance where the skill may cause runtime download/execution of an unpinned package version. While intended for convenience, it still exposes users to supply-chain compromise if the resolved package is malicious or tampered with.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
91% confidence
Finding

The coverage example npx vitest --coverage repeats the unpinned execution pattern and can pull a changing package version from the registry. Since this is reference material likely to be copied verbatim, it creates a practical supply-chain risk rather than a merely theoretical one.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
90% confidence
Finding

The npx jest --coverage example is vulnerable for the same reason: it may execute an unpinned version fetched at runtime. In an agent skill that encourages command execution, this can expose the environment to malicious package updates or inconsistent tool behavior.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.