Back to skill

Security audit

Paper to Pipeline

Security checks for vulnerabilities and agentic risk

Overview

This skill is a mostly coherent ML pipeline generator, but it can turn malicious experiment-plan text into executable Python code that users are instructed to run.

Review before installing. Use this only with experiment plans you trust, inspect generated files before running them, and install dependencies in a fresh virtual environment. Avoid running generated main.py until the generator escapes or validates plan-derived values in config.py and documents any remote model downloads.

Vulnerability Patterns
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (2)

T09 · Insecure Skill Coding Practices

Error
Location
scripts/generate_pipeline.py:276
Finding

Untrusted Experiment Plan Values Are Embedded into Executable Python Code

Content
View full analysis

Vulnerability Details

File Location: scripts/generate_pipeline.py, source extraction at lines 62–67, 75–79, and 89–96; vulnerable code-generation sink at lines 276–288
Vulnerability Type: Python code injection through unsafe string interpolation
Risk Level: High

Vulnerable Code

The generator extracts attacker-controlled strings directly from the experiment plan:

python
config['dataset'] = {
    'name': name_match.group(1).strip() if name_match else 'unknown',
    'type': type_match.group(1).strip() if type_match else 'unknown',
    'scale': scale_match.group(1).strip() if scale_match else 'unknown',
    'input_shape': input_match.group(1).strip() if input_match else None,
    'output_shape': output_match.group(1).strip() if output_match else None,
}
python
config['model'] = {
    'backbone': backbone_match.group(1).strip() if backbone_match else 'resnet18',
    'num_layers': int(layers_match.group(1)) if layers_match else None,
    'hidden_size': int(hidden_match.group(1)) if hidden_match else 128,
}
python
config['training'] = {
    'optimizer': optimizer_match.group(1).strip().lower() if optimizer_match else 'adamw',
    'learning_rate': float(lr_match.group(1)) if lr_match else 0.001,
    'batch_size': int(batch_match.group(1)) if batch_match else 32,
    'epochs': int(epochs_match.group(1)) if epochs_match else 100,
    'loss': loss_match.group(1).strip().lower() if loss_match else 'cross_entropy',
}

These strings are then embedded verbatim inside quoted Python literals:

python
class Config:
    # 任务类型
    task_type = "{config.get('task_type', 'classification')}"
    
    # 数据集配置
    dataset_name = "{dataset_info.get('name', 'unknown')}"
    dataset_type = "{dataset_info.get('type', 'image')}"
    num_classes = {num_classes}
    
    # 模型配置
    model_name = "{model_info.get('backbone', 'resnet18')}"
    pre
...[truncated 2593 chars]
Remediation
View remediation

Remediation Suggestions

  1. Do not construct Python string literals by directly interpolating untrusted text.

  2. Serialize every generated Python string with a safe representation:

    python
    dataset_name = repr(dataset_info.get('name', 'unknown'))
    dataset_type = repr(dataset_info.get('type', 'image'))
    model_name = repr(model_info.get('backbone', 'resnet18'))
    optimizer = repr(training_info.get('optimizer', 'adamw'))
    

    The template should then insert those already escaped literals without adding another pair of quotation marks.

  3. Prefer generating a non-executable configuration file such as YAML or JSON using yaml.safe_dump() or json.dump(), and load it as data rather than Python source.

  4. Apply strict allowlists to fields with a finite set of supported values:

    • Dataset type: supported dataset categories only.
    • Optimizer: adamw, adam, or sgd.
    • Model name: explicitly supported model identifiers.
    • Task type: explicitly supported task identifiers.
  5. Reject line breaks, control characters, and invalid syntax in fields intended to be simple identifiers.

  6. Add regression tests using quotation marks, backslashes, multiline values, comments, and Python expressions in every interpolated plan field.

  7. Treat generated code as untrusted until it has passed syntax validation and security review.

T08 · Insecure Dependencies

Warning
Location
scripts/generate_pipeline.py:908
Finding

Generated Dependency Manifest Uses Unbounded Mutable Version Ranges

Content
View full analysis

Vulnerability Details

File Location: scripts/generate_pipeline.py, lines 908–933; installation instructions at lines 946–949 and 1031
Vulnerability Type: Unlocked third-party dependencies and non-reproducible package installation
Risk Level: Medium

Vulnerable Code

python
def generate_requirements(config):
    """生成 requirements.txt"""
    base_deps = [
        'torch>=2.0.0',
        'torchvision>=0.15.0',
        'numpy>=1.24.0',
        'pandas>=2.0.0',
        'scikit-learn>=1.3.0',
        'tqdm>=4.65.0',
        'pyyaml>=6.0',
    ]
    
    dataset_type = config.get('dataset', {}).get('type', 'image')
    model_name = config.get('model', {}).get('backbone', '').lower()
    
    if dataset_type == 'text' or 'bert' in model_name:
        base_deps.extend([
            'transformers>=4.30.0',
            'tokenizers>=0.13.0',
        ])
    
    return '\n'.join(base_deps)

The generated README directs users to install these mutable requirements:

markdown
### 1. 安装依赖

```bash
pip install -r requirements.txt
text

The command-line completion message repeats the same instruction:

```python
print(f"  2. pip install -r requirements.txt")

Technical Analysis

Every generated dependency uses a lower-bound-only constraint. Consequently, the exact installed package versions and transitive dependency graph depend on what is available from the configured package index at installation time.

The generated environment is therefore not reproducible and may silently consume future releases that have not been reviewed or tested with the generated project. The installation command also does not require package hashes or identify an approved package index.

The audited code does not contain an explicitly malicious package name, a typosquatted dependency, or a suspicious package repository. The risk arises from mutable depend ...[truncated 1487 chars]

Remediation
View remediation

Remediation Suggestions

  1. Replace lower-bound-only dependency declarations with exact, reviewed versions.

  2. Produce a lock file that includes all transitive dependencies for supported Python versions and platforms.

  3. Include cryptographic hashes and recommend installation with hash verification, such as:

    bash
    pip install --require-hashes -r requirements.lock
    
  4. Generate dependency manifests through a reproducible tool such as pip-tools, Poetry, or another reviewed lock-file workflow.

  5. Document and enforce an approved HTTPS package index rather than relying silently on arbitrary user configuration.

  6. Run dependency vulnerability and provenance scanning as part of release and generation testing.

  7. Recommend installation in a dedicated, unprivileged virtual environment. Explicitly warn users not to run package installation as root or with unnecessary administrative privileges.

  8. Establish a controlled dependency-update process in which new versions are tested and reviewed before lock files are refreshed.

Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (12)

Tp4

High
Category
MCP Tool Poisoning
Confidence
96% confidence
Finding

The skill claims broad capability—parsing uploaded plans and generating full pipelines across PyTorch, TensorFlow, and scikit-learn—but the finding indicates the actual implementation is much narrower and incomplete. Security-relevant mismatches like this are dangerous because users and orchestrators may trust the skill with inputs, file access, or automation decisions under false assumptions, leading to unsafe execution, broken outputs, or misuse of generated code.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
89% confidence
Finding

The skill describes behavior that reads user-uploaded documents and generates multiple files on disk, but it does not declare any explicit tool or permission scope. This creates an authorization ambiguity: an agent may be granted broader file read/write capability than the user expects, increasing the risk of unintended file access or persistence beyond the intended workspace.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The instruction 关键代码段添加中文注释 forces a specific language in generated output. The file does not offer a language preference option or explain a region-specific requirement, so this is a natural-language locale policy violation.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The file’s natural-language docstring and user-facing messages are written only in Chinese, including the title and runtime output. This imposes a language choice without offering the user an alternative or documenting that the skill is region-specific.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
89% confidence
Finding

This markdown file presents the skill reference and instructions only in Chinese, which can constitute a language/locale policy violation when no user opt-in or alternative language option is provided. The document does not state that it is intended solely for a Chinese-language or region-specific context.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

For text datasets, the generated code calls BertTokenizer.from_pretrained('bert-base-chinese'), which typically fetches tokenizer assets from external repositories if not already cached. The manifest describes a skill that generates runnable ML pipeline code from an experiment plan, but does not justify adding network-dependent model download behavior as part of that purpose.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

This Python file generates a dataset module that always uses bert-base-chinese, and elsewhere also generates a BERT model defaulting to the same Chinese-only checkpoint. That imposes a specific language/locale behavior on downstream use without offering opt-in, selection, or explaining that the skill is limited to Chinese-language experiments.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The generated create_model function defaults model_name to bert-base-chinese, which bakes in a specific language choice for text classification workflows. Under the policy, forcing a locale/language without user opt-in is a natural-language policy violation unless clearly justified as region-specific.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The generated BERT path uses BertForSequenceClassification.from_pretrained(model_name), which commonly downloads model weights and configuration from remote hubs. While code generation is within scope, embedding automatic remote model retrieval is a separate capability not stated in the manifest for a paper-to-pipeline generator.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
82% confidence
Finding

This markdown file describes producing a concrete output directory with multiple files such as main.py, config.yaml, and README.md, which implies filesystem writes affecting user data or workspace state. Under the markdown-file criteria for SQP-2, the description should warn users about this behavior because it can modify the user's project directory.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
95% confidence
Finding

该 markdown 文档从标题到内容均固定为中文,没有说明这是特定区域/受众限定的材料,也没有提供其他语言选项。根据语言/区域政策,强制单一语言且缺少用户选择或明确合理性说明,属于自然语言层面的政策风险。

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Low
Category
Not specified by scanner
Confidence
84% confidence
Finding

The generated image-model code sets pretrained=True by default and passes torchvision weight enums into model constructors, which can trigger downloading pretrained weights when absent locally. The skill is described as generating experiment pipeline code, not provisioning external artifacts, so this additional capability is not clearly justified by the manifest.

Content

No source excerpt is available for this finding.