Back to skill

Security audit

LoRA Toolkit

Security checks for vulnerabilities and agentic risk

Overview

This LoRA training helper is mostly purpose-aligned, but it generates scripts that run unpinned remote model code and contains command-injection-prone handling of user inputs.

Review before installing or running. Use only trusted model repositories, disable trust_remote_code or pin reviewed revisions, run training in an isolated environment without sensitive credentials, and avoid passing untrusted filenames or model strings to this script until the input-handling bugs are fixed.

Vulnerability Patterns
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
Findings (4)

T09 · Insecure Skill Coding Practices

Error
Location
scripts/script.sh:292
Finding

Arbitrary Python Code Execution Through a Crafted Dataset Filename

Content
View full analysis
/dev/null || echo "parse-error") ``` ### Technical Analysis The user-controlled `--file` value is interpolated directly into Python source code enclosed by `open('$file')`. Shell quotation does not make this value safe for the Python parser. A filename containing a single quote, closing parentheses, semicolons, and additional Python statements can terminate the intended expression and inject arbitrary Python code. The preceding `[[ -f "$file" ]]` check does not prevent exploitation. An attacker who can create files in the working directory can create both the crafted filename required to pass that check and any secondary file referenced by the injected Python expression. The script already uses a safe argument-based approach in the subsequent heredoc invocation: ```bash python3 - "$file" "$fmt" << 'PYEOF' ``` The same design should be applied to the initial count operation. ### Attack Path 1. An attacker creates a JSON file and an additional file whose name contains Python syntax. 2. The attacker invokes: ```bash bash scripts/script.sh validate --file "" ``` 3. The shell expands `$file` into the `python3 -c` source string. 4. The crafted quote and Python syntax escape the intended `open()` call. 5. Python executes the injected statements with the privileges of the user running the Skill. For example, the injected statements could invoke `os.system()`, read accessible files, modify the user's files, or launch another executable. ### Impact Assessment Successful exploitation provides arbitrary code execution under the current user's account. The attacker can access any files, environment variables, tokens, SSH mate ...[truncated 226 chars]
Remediation
View remediation
/dev/null || echo "parse-error" ) ``` Alternatively, combine the counting and format validation operations into the existing argument-based Python heredoc. Never construct executable Python source using direct interpolation of user-controlled values. ]]>

T09 · Insecure Skill Coding Practices

Error
Location
scripts/script.sh:267
Finding

GNU sed Program Injection Through the Model Argument

Content
View full analysis
" --output train.py ``` 3. Expansion of `$model` changes the `sed` program passed to the executable. 4. On a compatible `sed` implementation, injected command-execution syntax may execute immediately. 5. Alternatively, attacker-controlled content is written into `train.py`. 6. The user follows the generated instructions and runs `python3 train.py`, executing the injected Python content. ### Impact Assessment Immediate exploitation through implementation-specific `sed` extensions can execute commands with the priv ...[truncated 278 chars]
Remediation
View remediation

T03 · Remote Payload Retrieval and Execution

Error
Location
scripts/script.sh:219
Finding

Generated Training Script Executes Unpinned Remote Model Code

Content
View full analysis
Remediation
View remediation

T08 · Insecure Dependencies

Warning
Location
scripts/script.sh:186
Finding

Unpinned Machine-Learning Dependencies Create Supply-Chain Exposure

Content
View full analysis
Remediation
View remediation
Vulnerability Patterns
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Findings (11)

Tp4

High
Category
MCP Tool Poisoning
Confidence
95% confidence
Finding

The core declared purpose is largely accurate for the main commands: config, estimate, and generate align well with configuring LoRA fine-tuning, estimating training cost, and generating a PEFT-style training script. However, the actual code includes two extra user-facing commands beyond the declaration: validate, which reads and parses a local dataset file to inspect format and sample count, and recommend, which suggests models based on task, GPU, and language. These are substantive undeclared capabilities, not merely internal helpers. Because the evaluation criteria say to flag undeclared capabilities when code performs them, this should be marked as a mismatch, even though the primary purpose is mostly aligned.

Content

No source excerpt is available for this finding.

External Model or Provider Selection

High
Category
Excessive Agency
Confidence
90% confidence
Finding

Skill selects an external model or provider that may use a different account or billing plan than the operator expects. Undisclosed model switches can cause unexpected cost or quota consumption.

Content

Scanner excerpt · SKILL.md (reported line 18)May include surrounding context.

config

Generate a LoRA training configuration for your model and hardware.

bash
bash scripts/script.sh config --model llama3-8b --gpu 24gb --dataset 10000

Parameters:

  • --model — base model (llama3-8b, mistral-7b, qwen2-7b, phi3-mini, llama3-70b)

External Model or Provider Selection

High
Category
Excessive Agency
Confidence
90% confidence
Finding

Skill selects an external model or provider that may use a different account or billing plan than the operator expects. Undisclosed model switches can cause unexpected cost or quota consumption.

Content

Scanner excerpt · scripts/script.sh (reported line 409)May include surrounding context.

sh
### config
Generate a LoRA training configuration for your model and hardware.
```bash
bash scripts/script.sh config --model llama3-8b --gpu 24gb --dataset 10000
```
Parameters:
- `--model` — base model (llama3-8b, mistral-7b, qwen2-7b, phi3-mini, llama3-70b)

External Model or Provider Selection

High
Category
Excessive Agency
Confidence
90% confidence
Finding

Skill selects an external model or provider that may use a different account or billing plan than the operator expects. Undisclosed model switches can cause unexpected cost or quota consumption.

Content

Scanner excerpt · SKILL.md (reported line 28)May include surrounding context.

estimate

Estimate VRAM usage, training time, and cost before starting.

bash
bash scripts/script.sh estimate --model mistral-7b --gpu 16gb --dataset 5000 --epochs 3

generate

External Model or Provider Selection

High
Category
Excessive Agency
Confidence
90% confidence
Finding

Skill selects an external model or provider that may use a different account or billing plan than the operator expects. Undisclosed model switches can cause unexpected cost or quota consumption.

Content

Scanner excerpt · scripts/script.sh (reported line 410)May include surrounding context.

sh
### estimate
Estimate VRAM usage, training time, and cost before starting.
```bash
bash scripts/script.sh estimate --model mistral-7b --gpu 16gb --dataset 5000 --epochs 3
```

### generate

External Model or Provider Selection

High
Category
Excessive Agency
Confidence
90% confidence
Finding

Skill selects an external model or provider that may use a different account or billing plan than the operator expects. Undisclosed model switches can cause unexpected cost or quota consumption.

Content

Scanner excerpt · SKILL.md (reported line 34)May include surrounding context.

generate

Generate a ready-to-run Python training script using HuggingFace PEFT + TRL.

bash
bash scripts/script.sh generate --model llama3-8b --output train.py

validate

External Model or Provider Selection

High
Category
Excessive Agency
Confidence
90% confidence
Finding

Skill selects an external model or provider that may use a different account or billing plan than the operator expects. Undisclosed model switches can cause unexpected cost or quota consumption.

Content

Scanner excerpt · scripts/script.sh (reported line 411)May include surrounding context.

sh
### generate
Generate a ready-to-run Python training script using HuggingFace PEFT + TRL.
```bash
bash scripts/script.sh generate --model llama3-8b --output train.py
```

### validate

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
98% confidence
Finding

The generated Python training script loads Hugging Face model and tokenizer code with trust_remote_code=True, which allows arbitrary Python from a model repository to execute locally during model loading. In this skill's context, users are explicitly encouraged to generate and run the script, so a malicious or compromised model repo could achieve code execution on the training host.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
85% confidence
Finding

The generated Python script loads a dataset and saves trained model artifacts to config.output_dir via model.save_pretrained and tokenizer.save_pretrained, which can create or overwrite files on disk. While the shell script logs that it is generating a file, there is no explicit warning to the user that running the generated trainer will write checkpoints and adapters to local storage.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Low
Category
Not specified by scanner
Confidence
85% confidence
Finding

The manifest describes a tool that configures, estimates, and generates LoRA fine-tuning scripts from model, dataset size, and GPU spec. The code also implements dataset-format validation and base-model recommendation features, which are not mentioned in the manifest and expand the skill's behavior beyond its declared purpose.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
77% confidence
Finding

The recommendation logic defaults to lang="en" and emits language-targeted advice such as 'best for zh/en/ja' and a special 'Chinese tip'. This is a mild natural-language policy concern because the skill bakes in language assumptions rather than clearly presenting language selection as an explicit user opt-in choice in the help or output.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.