AB Test Framework

• Outcome: Review
• Latest audit: Sep 11, 2026 · 12:41 AM UTC
• Version: 1.0.0

Overview

Review recommended because this skill can store submitted test prompts in logs and has a weak, malformed dependency setup.

Install only after reviewing whether prompt contents may enter logs in your environment. The publisher should remove or pin the unused dependency with a lockfile, fix package.json, redact logged inputs, and implement the documented A/B testing behavior before relying on this skill for real model evaluation.

A.I.G

Vulnerability Patterns

• Insecure Dependencies: Introduces malicious components through unsafe dependency sources
• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review

Findings (2)

T09 · Insecure Skill Coding Practices

Warning

• Location: index.js:28
• Finding: Unsanitized User-Controlled Parameters Written to Logs<![CDATA[ ## Vulnerability Details **File Location**: `index.js`, line 28 **Vulnerability Type**: Sensitive data exposure and log injection **Risk Level**: Medium ### Vulnerable Code ```javascript logger.info(`[ab-test-framework] Starting execution`, { params }); ``` ### Technical Analysis The skill writes the complete caller-controlled `params` object to the application log before invoking `validate.sanitize(params)` on line 41. The logged object can include model identifiers, complete test prompts, confidential business information, personal data, or attacker-supplied control characters. Because sanitization occurs only after this logging operation, it does not protect the logged data. Depending on the logger and downstream log-processing system, control characters or specially formatted values could also forge log entries or manipulate how records are parsed. This behavior contradicts the documented claim that inputs are validated before processing. ### Attack Path 1. An attacker or untrusted caller submits sensitive or crafted content through `model_a`, `model_b`, `test_prompts`, or additional properties accepted in `params`. 2. The skill passes the complete unsanitized object to `logger.info`. 3. The logging system stores or forwards the raw values. 4. Users, services, or third-party monitoring platforms with access to those logs can retrieve the submitted content. 5. If the logging pipeline inadequately escapes control characters, crafted input may create misleading entries or interfere with downstream parsing and alerting. ### Impact Assessment The issue does not directly grant operating-system privileges. Its scope is the confidentiality and integrity of the application's logging environment. Potential consequences include: - Disclosure of confidential prompts or other caller-supplied information. - Exposure to administrators, support staff, monitoring services, or other log consumers that would not otherwise receive prompt contents. - Log-fo ...[truncated 200 chars]
• Remediation: <![CDATA[ ## Remediation Suggestions - Do not log complete input objects or prompt contents. - Log only allowlisted, non-sensitive metadata, such as model identifiers, prompt count, request ID, and execution duration. - Sanitize model identifiers and encode control characters before logging. - Apply field-level redaction in the logger as a defense-in-depth measure. - Establish retention and access controls for operational logs. - Add tests confirming that prompt contents and unknown input properties never appear in logs. For example: ```javascript const modelA = validate.sanitize(params.model_a); const modelB = validate.sanitize(params.model_b); const promptCount = Array.isArray(params.test_prompts) ? params.test_prompts.length : 0; logger.info('[ab-test-framework] Starting execution', { model_a: modelA, model_b: modelB, prompt_count: promptCount }); ``` ]]>

T08 · Insecure Dependencies

Warning

• Location: package.json:6
• Finding: Unnecessary Mutable Third-Party Dependency Without an Integrity Lock<![CDATA[ ## Vulnerability Details **File Location**: `package.json`, lines 6-8 **Vulnerability Type**: Third-party software supply-chain exposure **Risk Level**: Medium ### Vulnerable Code ```json "dependencies": { "stats-library": "^1.0.0" } ``` The project contains no package lockfile in the audited directory structure, and the dependency is not imported or used by `index.js`. ### Technical Analysis The caret version range permits npm to select later semver-compatible releases instead of requiring the exact version originally reviewed. Without a committed lockfile and its integrity metadata, separate installations may resolve to different package artifacts. The dependency is currently unused, so it increases the installation attack surface without supporting implemented behavior. If the package, a permitted future release, or its publication account were compromised, installation could introduce malicious package files or lifecycle scripts. This finding does not establish that the current `stats-library` package is malicious. It identifies avoidable supply-chain exposure caused by an unused, broadly versioned dependency and the absence of a lockfile. ### Attack Path 1. A user follows the project documentation and runs `npm install`. 2. npm resolves a release satisfying `^1.0.0`; without a lockfile, the resolved artifact can differ across installations. 3. An attacker compromises the package publication channel or publishes a malicious semver-compatible release through a compromised maintainer account. 4. npm downloads the compromised artifact. 5. Any malicious lifecycle script permitted by the installation environment executes with the privileges of the user or automation account running npm; malicious runtime code could also execute if the dependency is later imported. ### Impact Assessment The obtainable privileges are limited to those of the account and environment performing package installation or running imported dependency code. In a devel ...[truncated 542 chars]
• Remediation: <![CDATA[ ## Remediation Suggestions - Remove `stats-library` because the current implementation does not use it. - If statistical functionality is implemented later, review the package's ownership, release history, transitive dependencies, and lifecycle scripts before adoption. - Pin the dependency to an exact reviewed version rather than a caret range. - Generate and commit a package lockfile containing resolved versions and integrity hashes. - Use deterministic installation in automation, such as `npm ci`. - Run dependency installation with minimal privileges and without access to production secrets. - Disable lifecycle scripts where compatible with the build process, for example through `npm ci --ignore-scripts`. - Add automated dependency vulnerability and provenance checks to CI. ]]>

SkillSpector

Vulnerability Patterns

• Supply Chain: Unpinned Dependencies, External Script Fetching, Obfuscated Code
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access
• Excessive Agency: Unrestricted Tool Access, Autonomous Decision Making, Scope Creep

Findings (1)

Unpinned Dependencies

Low

• Category: Supply Chain

Content

"description": "Compare models with A/B testing for selection",
  "main": "index.js",
  "dependencies": {
    "stats-library": "^1.0.0"
  }}
}

• Confidence: 93% confidence
• Finding: The dependency uses a caret range (^1.0.0), which permits automatic installation of newer compatible releases rather than an exact vetted version. This can introduce supply-chain risk if a later published package version is compromised, vulnerable, or behaviorally incompatible, though the impact here is limited because this file alone does not show privileged install hooks or especially sensitive dependency usage.

Static analysis

No suspicious patterns detected.