Back to skill

Security audit

Backtest Free

Security checks for vulnerabilities and agentic risk

Overview

This finance backtesting skill is mostly coherent, but it needs review because it allows broad command execution and unpinned package installs while producing investment-style recommendations.

Install only in an isolated virtual environment or container, avoid administrator/root execution, verify the backtest-free package source and versions before use, and treat any trading signals or forecasts as informational backtest output rather than investment advice.

Vulnerability Patterns
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T08 · Insecure Dependencies

Warning
Location
SKILL.md:121
Finding

Unpinned Third-Party Packages Enable Supply-Chain Compromise

Content
View full analysis

Vulnerability Details

File Location: SKILL.md, lines 121–127; duplicated installation guidance at line 42
Vulnerability Type: Unpinned and externally supplied Python dependencies
Risk Level: Medium

Vulnerable Code Snippet

text
Python version: 3.7+
Dependency installation command: pip install backtest-free pandas numpy matplotlib

1. Import library: import backtest_free
2. Create strategy: strategy = backtest_free.create_strategy()
3. Set parameters: strategy.set_parameters(stock_code='AAPL', period='1m', risk_threshold=0.05)
4. Run backtest: results = strategy.run_backtest()
5. Analyze results: results.analyze()

The same unpinned installation command is also presented in the troubleshooting guidance:

shell
pip install backtest-free pandas numpy matplotlib

Technical Analysis

The Skill directs users to install backtest-free, pandas, numpy, and matplotlib from pip without exact version constraints, package hashes, a lockfile, or publisher verification. Consequently, the artifacts installed at execution time may differ from those reviewed during the Skill audit.

This concern is particularly significant for backtest-free, because the repository contains only SKILL.md and no local implementation of the advertised backtesting functionality. The externally installed package therefore supplies the operative code. Importing backtest_free and invoking its methods executes package-controlled Python code with the permissions of the user running the agent.

The audit did not establish that any currently published package is malicious. The confirmed weakness is that the documented installation process does not provide dependency integrity or reproducibility, leaving it exposed to package compromise, malicious future releases, dependency confusion, or transitive dependency attacks.

Attack Path

  1. An attacker compromises an upstream package, publisher account, or trans ...[truncated 1598 chars]
Remediation
View remediation

Remediation Suggestions

  1. Pin every direct dependency to an exact, reviewed version rather than allowing unrestricted resolution.
  2. Generate a lockfile containing the complete transitive dependency graph.
  3. Require package hashes, for example through a hash-locked requirements file and pip install --require-hashes.
  4. Document the authoritative publisher, source repository, package index, and expected package signatures or checksums for backtest-free.
  5. Vendor or include the auditable implementation in the Skill package when feasible, instead of delegating all behavior to mutable external code.
  6. Review dependency source code and build metadata before approving each update. Use automated vulnerability, malware, and provenance scanning.
  7. Install dependencies inside an isolated virtual environment or container with restricted filesystem and network access.
  8. Run the Skill as a dedicated, non-administrative account. Remove the recommendation to retry with administrator privileges.
  9. Restrict or remove the exec capability unless it is necessary for a clearly documented operation, and allow only fixed commands and arguments.
  10. Add a reproducible installation example, such as:
shell
python -m pip install --require-hashes -r requirements.lock
Vulnerability Patterns
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (5)

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
87% confidence
Finding

The display name, summary, and description are written entirely in Chinese, with no indication that users can choose another language or that the skill is intentionally limited to a Chinese-speaking region. Under the locale policy, forcing a specific language without opt-in can be a natural-language policy violation.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The skill explicitly claims that risky code was removed and safety was improved, yet it still declares the exec tool. That mismatch is dangerous because it can create false trust for users or calling agents while retaining the ability to run shell commands, which materially expands attack surface if later prompts or workflows pass attacker-controlled input.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The activation language is very broad and says to use the skill whenever users need related functionality, without strong exclusions. Overbroad routing can cause the skill to activate on generic finance requests, increasing the chance that high-risk capabilities or authoritative-seeming investment outputs are used outside intended boundaries.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

A financial backtesting/documentation skill does not obviously require arbitrary command execution, so exposing exec is an unjustified privilege. Excess capability increases the chance that prompt injection, misrouting, or unsafe future implementation details could trigger local command execution unrelated to the user’s legitimate task.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The skill provides trading analysis, signal generation, and investment-oriented recommendations but does not include a clear warning that outputs are not investment advice. This can mislead users into treating generated content as professional financial advice, creating legal, compliance, and user-harm risk, especially when recommendations are presented with confidence and quantified performance claims.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.