ML Model Eval Benchmark

• Outcome: Pass
• Latest audit: Sep 11, 2026 · 12:53 AM UTC
• Version: 0.1.0

Overview

This skill is a small local model-ranking helper with disclosed file input and output behavior, though users should treat CSV exports and dry-run mode carefully.

Install only if you are comfortable with a local script writing benchmark output files. Use trusted or reviewed input data, avoid opening CSV exports from untrusted model names in spreadsheet software, and do not rely on --dry-run to prevent file creation or overwrite unless the script is fixed.

A.I.G

Vulnerability Patterns

• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review
• Embedded Malicious Code: Ships malicious scripts inside the skill and executes them locally

Findings (2)

T09 · Insecure Skill Coding Practices

Warning

• Location: scripts/benchmark_models.py:61

Finding

Spreadsheet Formula Injection Through Untrusted Model Names in CSV Output

Content

View full analysis

Vulnerability Details

File Location: scripts/benchmark_models.py, lines 61-64; untrusted value introduced at line 80
Vulnerability Type: CSV/spreadsheet formula injection
Risk Level: Medium

Vulnerable Code

python

with output_path.open("w", newline="", encoding="utf-8") as handle:
    writer = csv.DictWriter(handle, fieldnames=["rank", "name", "score"])
    writer.writeheader()
    writer.writerows(result["details"]["leaderboard"])

The model name included in the exported rows is obtained directly from input:

python

name = str(model.get("name", "unknown-model"))

Technical Analysis

Model names originate from an externally supplied JSON document and are copied into the leaderboard without validation or spreadsheet-specific output encoding. Although csv.DictWriter correctly escapes CSV delimiters and quotes, CSV escaping does not neutralize spreadsheet formulas.

If a model name starts with a formula indicator such as =, +, -, or @, spreadsheet software may interpret the cell as a formula when a user opens the generated CSV file. The exact behavior and available formula capabilities depend on the spreadsheet product and its security configuration.

Attack Path

1. An attacker supplies or influences the benchmark input JSON.
2. The attacker sets a model name to a formula-like value, such as =HYPERLINK("https://attacker.example","Open report").
3. An operator runs the script with --format csv.
4. The model name passes through str() without formula neutralization.
5. csv.DictWriter writes the attacker-controlled value into the name column.
6. A user opens the resulting file in spreadsheet software.
7. The application may evaluate or display the value as a formula, potentially enabling deceptive links or other spreadsheet-supported actions.

Impact Assessment

The vulnerability does not grant privileges in the Python process and does not itself e ...[truncated 471 chars]

Remediation

View remediation

Remediation Suggestions

Treat all text fields exported to CSV as untrusted. Before writing a cell, detect values whose first non-whitespace character is =, +, -, or @ and neutralize them using a strategy compatible with the intended spreadsheet consumers, such as prefixing an apostrophe.

Example:

python

def sanitize_csv_cell(value: object) -> object:
    if not isinstance(value, str):
        return value
    if value.lstrip().startswith(("=", "+", "-", "@")):
        return "'" + value
    return value

Apply the function to model names before passing rows to DictWriter. Add tests for formula prefixes, leading whitespace, tabs, delimiters, quotes, and multiline values. Document that candidate names are untrusted and that CSV artifacts should not be allowed to contain active spreadsheet content.

T09 · Insecure Skill Coding Practices

Note

• Location: scripts/benchmark_models.py:17

Finding

Dry-Run Mode Performs Filesystem Writes Despite Claiming No Side Effects

Content

View full analysis

Vulnerability Details

File Location: scripts/benchmark_models.py, lines 17 and 107
Vulnerability Type: Unsafe dry-run implementation and unexpected file modification
Risk Level: Low

Vulnerable Code

The command-line option promises operation without side effects:

python

parser.add_argument("--dry-run", action="store_true", help="Run without side effects.")

However, output rendering is unconditional:

python

render(result, Path(args.output), args.format)

The rendering function creates directories and writes the output:

python

def render(result: dict, output_path: Path, fmt: str) -> None:
    output_path.parent.mkdir(parents=True, exist_ok=True)

    if fmt == "json":
        output_path.write_text(json.dumps(result, indent=2), encoding="utf-8")
        return

Technical Analysis

The --dry-run flag is recorded in the result but is never used to suppress filesystem operations. Consequently, a dry run still creates missing parent directories and creates or overwrites the specified output file.

This violates the option's explicit no-side-effects contract. Operators and automation commonly rely on dry-run modes to validate inputs and planned actions safely. The mismatch can therefore cause unintended modification of files accessible to the invoking user.

Attack Path

1. An operator or automated process invokes the script with --dry-run, expecting no filesystem changes.
2. The invocation supplies an output path that already exists or whose parent directory does not yet exist.
3. The script computes the benchmark and unconditionally calls render().
4. render() creates parent directories and writes or overwrites the output file.
5. Data at the selected path may be modified despite the operator's reliance on dry-run semantics.

Exploitation requires control or influence over the command arguments, especially --output, or operator selection of a s ...[truncated 541 chars]

Remediation

View remediation

Remediation Suggestions

Enforce dry-run behavior before invoking any function that creates directories or writes files:

python

if not args.dry_run:
    render(result, Path(args.output), args.format)

If dry-run results must be visible, print them to standard output without touching the filesystem. Alternatively, remove or revise the “without side effects” claim if writing output is intentionally part of dry-run behavior.

Add automated tests asserting that, with --dry-run, no output file or parent directory is created and existing output files remain unchanged. For normal execution, consider refusing to overwrite existing files unless an explicit --force option is supplied.

SkillSpector

Vulnerability Patterns

• MCP Least Privilege: Underdeclared Capability, Wildcard Permission, Missing Permission Declaration
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access
• Supply Chain: Unpinned Dependencies, External Script Fetching, Obfuscated Code

Findings (1)

Undeclared Tool Scope

Medium

• Category: MCP Least Privilege
• Confidence: 88% confidence

Finding

The skill references bundled executable resources and file-based inputs/outputs, but the manifest does not declare any explicit tool scope such as permissions or allowed-tools. This creates an authorization ambiguity where an agent may grant broader file read/write capability than intended, increasing the risk of unintended filesystem access or misuse by downstream automation.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.