Backtest Engine - Run Backtests

• Outcome: Review
• Latest audit: Sep 14, 2026 · 11:17 PM UTC
• Version: 0.1.0

Overview

The skill is a real backtesting tool, but it can execute user-supplied Python strategy files without a clear warning or sandbox.

Install only if you understand that custom strategy files are Python programs and should be treated like any other code you run locally. Use built-in strategies or trusted, reviewed strategy files, run in an isolated virtual environment, and avoid exposing secrets or sensitive workspace files when testing third-party strategies.

A.I.G

Vulnerability Patterns

• Insecure Dependencies: Introduces malicious components through unsafe dependency sources
• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review

Findings (2)

T09 · Insecure Skill Coding Practices

Error

• Location: scripts/backtest_engine.py:86

Finding

Unrestricted Execution of User-Supplied Python Strategy Files

Content

View full analysis

Vulnerability Details

File Location: scripts/backtest_engine.py, lines 86–91
Vulnerability Type: Arbitrary code execution through an untrusted plugin
Risk Level: High

Vulnerable Code:

python

def load_custom_strategy(strategy_file: str, strategy_fn: str) -> Callable:
    """Load a user-defined strategy function from an external file."""
    spec = importlib.util.spec_from_file_location("user_strategy", strategy_file)
    mod = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(mod)
    if not hasattr(mod, strategy_fn):

Technical Analysis

The --strategy-file argument is treated as a Python module and executed using exec_module(). Importing a Python module executes all of its top-level statements before the engine checks whether the requested strategy function exists.

The supplied file is not subject to an allowlist, integrity check, signature verification, trust confirmation, filesystem restriction, or execution sandbox. Consequently, this is not merely data parsing: it grants the strategy file the ability to execute arbitrary Python code with the same operating-system permissions and environment as the backtest process.

This behavior may be acceptable for explicitly trusted plugins, but the CLI and documentation do not establish a security boundary or clearly warn that a strategy file is fully executable code.

Attack Path

1. An attacker creates a Python strategy file containing malicious top-level statements.
2. The attacker persuades a user or agent to download or otherwise use that file in a backtest.
3. The user invokes the engine with --strategy-file malicious_strategy.py.
4. load_custom_strategy() passes the attacker-controlled path to spec_from_file_location().
5. spec.loader.exec_module(mod) immediately executes the file's top-level code.
6. The malicious code can perform actions under the identity and permissions of the backtest process, eve ...[truncated 752 chars]

Remediation

View remediation

Remediation Suggestions

• Prefer a declarative strategy format with a strictly validated schema rather than executable Python plugins.
• If Python plugins remain supported, explicitly describe them as trusted executable code and require affirmative confirmation before loading files from untrusted or newly downloaded locations.
• Execute plugins in a separate, least-privileged process or container rather than inside the main engine.
• Disable network access and restrict filesystem visibility for the plugin process where feasible.
• Apply operating-system resource limits for CPU time, memory, process creation, and output size.
• Restrict plugin paths to an administrator- or user-approved directory and reject symbolic-link escapes.
• Maintain an allowlist of approved plugin hashes or require signed plugins in managed deployments.
• Pass only the minimum required price data into the isolated process and validate the returned signal structure, dimensions, values, and size.
• Update SKILL.md and the CLI help to state clearly that --strategy-file executes arbitrary Python and must only be used with trusted files.

T08 · Insecure Dependencies

Warning

• Location: README.md:21

Finding

Unpinned Third-Party Dependencies Installed Without Integrity Verification

Content

View full analysis

Vulnerability Details

File Location: README.md, lines 21–23
Vulnerability Type: Uncontrolled third-party dependency resolution
Risk Level: Medium

Vulnerable Code:

bash

# Install dependencies
pip install yfinance pandas numpy scipy

Technical Analysis

The installation command resolves mutable package versions from the package index configured for pip. It does not constrain versions, use a lockfile, verify artifact hashes, or identify a trusted package index.

As a result, installations are not reproducible and may retrieve dependency versions that were never reviewed with this project. A compromised package-maintainer account, package-index compromise, malicious transitive dependency, or unexpected incompatible release could introduce code that executes during installation or when imported by the engine.

No evidence was found that the named packages are intentionally malicious. The risk arises from the absence of version and integrity controls rather than from a confirmed malicious package.

Attack Path

1. A user follows the dependency installation command in the README.
2. pip queries its configured index and resolves the latest compatible versions and transitive dependencies.
3. An attacker compromises a relevant package release, maintainer account, dependency, or package source.
4. The installation retrieves the attacker-controlled artifact because no expected version or hash is enforced.
5. Malicious code executes during package installation or when the backtest engine imports the affected package.

Impact Assessment

Compromised dependency code would execute with the permissions of the user running pip or the backtest engine. It could access files, environment variables, and network resources available to that process and could alter backtest behavior or reports.

The potential scope is broad because dependencies execute as ordinary Python code. Actual impact depends ...[truncated 119 chars]

Remediation

View remediation

Remediation Suggestions

• Provide a reviewed dependency lockfile containing exact direct and transitive versions.
• Publish a hash-pinned requirements file and install it with:

  bash

  pip install --require-hashes -r requirements.txt
  

• Configure dependency update automation to open reviewed changes rather than resolving mutable versions during routine installation.
• Document the trusted package index and avoid untrusted extra indexes.
• Run dependency vulnerability and provenance checks in continuous integration.
• Install dependencies inside an isolated virtual environment using a non-privileged account.
• Remove scipy from the installation instructions if it is not required, thereby reducing the supply-chain attack surface.

SkillSpector

Vulnerability Patterns

• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Excessive Agency: Unrestricted Tool Access, Autonomous Decision Making, Scope Creep
• Trigger Abuse: Overly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
• Behavioral AST: exec() Call, eval() Call, Dynamic Import
• MCP Least Privilege: Underdeclared Capability, Wildcard Permission, Missing Permission Declaration

Findings (12)

Tp4

High

• Category: MCP Tool Poisoning
• Confidence: 96% confidence

Finding

The skill claims to be an execution engine with concrete backtesting, data loading, optimization, and metric computation capabilities, but the file only provides descriptive instructions and examples. This mismatch can mislead an agent or user into trusting nonexistent analytical rigor, causing unsafe financial decision-making, incorrect autonomy assumptions, or downstream workflows that act on fabricated or incomplete results.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

High

• Category: Not specified by scanner
• Confidence: 99% confidence

Finding

The custom strategy loader imports and executes a user-supplied Python file with spec.loader.exec_module(mod), which runs arbitrary top-level code immediately. In an agent skill context, this can lead to arbitrary code execution, including filesystem access, network calls, credential theft, or process spawning, well beyond 'backtesting' behavior.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium

• Category: MCP Least Privilege
• Confidence: 70% confidence

Finding

Without declared permissions the skill's intent is opaque and cannot be validated.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium

• Category: Not specified by scanner
• Confidence: 93% confidence

Finding

The listed triggers include generic phrases like "test this strategy" and "generate performance metrics," which can plausibly appear in ordinary conversation and may match requests outside this specific backtest engine. The file also provides no negative examples or exclusion conditions to clarify when the skill should not activate.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium

• Category: Not specified by scanner
• Confidence: 81% confidence

Finding

The built-in strategy loader dynamically executes a local Python module at runtime instead of importing fixed internal functions. If an attacker can modify or replace strategy_templates.py in the skill directory or package path, this becomes code execution through a trusted workflow and undermines the assumption that 'built-in' strategies are safe.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 97% confidence

Finding

The tool provides a feature that executes arbitrary Python from a user-provided file but does not clearly warn users that supplying --strategy-file will run code on the host. This increases the chance of unsafe use, especially in an automated agent environment where users may assume they are only loading 'strategy logic' rather than granting code execution.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium

• Category: Not specified by scanner
• Confidence: 96% confidence

Finding

The docstring says long entry occurs when 'price > lower band' while parenthetically describing 'price < mean - entry_z * std', and short entry occurs when 'price < upper band' while parenthetically describing 'price > mean + entry_z * std'. The implementation at L151-L156 follows the parenthetical conditions via z-score thresholds, so the prose conditions in the docstring actively contradict the actual logic.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low

• Category: Not specified by scanner
• Confidence: 84% confidence

Finding

The README states that the skill uses yfinance and requires an internet connection, which implies outbound network requests to a third-party data source. However, it does not explicitly warn users that invoking the skill will transmit ticker/query information externally or that network access occurs during use.

Content

No source excerpt is available for this finding.

Dynamic attribute access via getattr()

Low

• Category: Dangerous Code Execution
• Confidence: 50% confidence

Finding

Dynamic getattr() with a non-literal attribute name can access arbitrary object attributes, potentially bypassing access controls.

Content

Scanner excerpt · scripts/backtest_engine.py (reported line 84)May include surrounding context.

python

fn_name = fn_map.get(name.lower(), name)
    if not hasattr(mod, fn_name):
        sys.exit(f"ERROR: Strategy '{fn_name}' not found in strategy_templates.py")
    return getattr(mod, fn_name)


def load_custom_strategy(strategy_file: str, strategy_fn: str) -> Callable:

Dynamic attribute access via getattr()

Low

• Category: Dangerous Code Execution
• Confidence: 50% confidence

Finding

Dynamic getattr() with a non-literal attribute name can access arbitrary object attributes, potentially bypassing access controls.

Content

Scanner excerpt · scripts/backtest_engine.py (reported line 94)May include surrounding context.

python

spec.loader.exec_module(mod)
    if not hasattr(mod, strategy_fn):
        sys.exit(f"ERROR: Function '{strategy_fn}' not found in {strategy_file}")
    return getattr(mod, strategy_fn)


# ──────────────────────────────────────────────────────────────────────────────

Missing User Warnings

Low

• Category: Not specified by scanner
• Confidence: 84% confidence

Finding

The function creates the output directory and writes JSON and Markdown report files. While this behavior is part of the tool's reporting purpose, the code itself does not provide a direct runtime disclosure when the write occurs, and users may not realize files will be created or overwritten in the chosen output path.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low

• Category: Not specified by scanner
• Confidence: 77% confidence

Finding

This Python file includes a live HTTP/data-download operation via yfinance when run as a script. Although it prints that it is fetching market data, there is no explicit warning that executing the file will contact an external service and transmit request metadata, which qualifies as a network operation requiring disclosure.

Content

No source excerpt is available for this finding.