Back to skill

Security audit

Paper to Pipeline

Security checks for vulnerabilities and agentic risk

Overview

This skill is a mostly coherent ML pipeline generator, but it can turn malicious experiment-plan text into executable Python code that users are instructed to run.

Review before installing. Use this only with experiment plans you trust, inspect generated files before running them, and install dependencies in a fresh virtual environment. Avoid running generated main.py until the generator escapes or validates plan-derived values in config.py and documents any remote model downloads.

Vulnerability Patterns
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (2)

T09 · Insecure Skill Coding Practices

Error
Location
scripts/generate_pipeline.py:276
Finding
Untrusted Experiment Plan Values Are Embedded into Executable Python Code## Vulnerability Details **File Location**: `scripts/generate_pipeline.py`, source extraction at lines 62–67, 75–79, and 89–96; vulnerable code-generation sink at lines 276–288 **Vulnerability Type**: Python code injection through unsafe string interpolation **Risk Level**: High ### Vulnerable Code The generator extracts attacker-controlled strings directly from the experiment plan: ```python config['dataset'] = { 'name': name_match.group(1).strip() if name_match else 'unknown', 'type': type_match.group(1).strip() if type_match else 'unknown', 'scale': scale_match.group(1).strip() if scale_match else 'unknown', 'input_shape': input_match.group(1).strip() if input_match else None, 'output_shape': output_match.group(1).strip() if output_match else None, } ``` ```python config['model'] = { 'backbone': backbone_match.group(1).strip() if backbone_match else 'resnet18', 'num_layers': int(layers_match.group(1)) if layers_match else None, 'hidden_size': int(hidden_match.group(1)) if hidden_match else 128, } ``` ```python config['training'] = { 'optimizer': optimizer_match.group(1).strip().lower() if optimizer_match else 'adamw', 'learning_rate': float(lr_match.group(1)) if lr_match else 0.001, 'batch_size': int(batch_match.group(1)) if batch_match else 32, 'epochs': int(epochs_match.group(1)) if epochs_match else 100, 'loss': loss_match.group(1).strip().lower() if loss_match else 'cross_entropy', } ``` These strings are then embedded verbatim inside quoted Python literals: ```python class Config: # 任务类型 task_type = "{config.get('task_type', 'classification')}" # 数据集配置 dataset_name = "{dataset_info.get('name', 'unknown')}" dataset_type = "{dataset_info.get('type', 'image')}" num_classes = {num_classes} # 模型配置 model_name = "{model_info.get('backbone', 'resnet18')}" pre ...[truncated 2593 chars]
Remediation
## Remediation Suggestions 1. Do not construct Python string literals by directly interpolating untrusted text. 2. Serialize every generated Python string with a safe representation: ```python dataset_name = repr(dataset_info.get('name', 'unknown')) dataset_type = repr(dataset_info.get('type', 'image')) model_name = repr(model_info.get('backbone', 'resnet18')) optimizer = repr(training_info.get('optimizer', 'adamw')) ``` The template should then insert those already escaped literals without adding another pair of quotation marks. 3. Prefer generating a non-executable configuration file such as YAML or JSON using `yaml.safe_dump()` or `json.dump()`, and load it as data rather than Python source. 4. Apply strict allowlists to fields with a finite set of supported values: - Dataset type: supported dataset categories only. - Optimizer: `adamw`, `adam`, or `sgd`. - Model name: explicitly supported model identifiers. - Task type: explicitly supported task identifiers. 5. Reject line breaks, control characters, and invalid syntax in fields intended to be simple identifiers. 6. Add regression tests using quotation marks, backslashes, multiline values, comments, and Python expressions in every interpolated plan field. 7. Treat generated code as untrusted until it has passed syntax validation and security review.

T08 · Insecure Dependencies

Warning
Location
scripts/generate_pipeline.py:908
Finding
Generated Dependency Manifest Uses Unbounded Mutable Version Ranges## Vulnerability Details **File Location**: `scripts/generate_pipeline.py`, lines 908–933; installation instructions at lines 946–949 and 1031 **Vulnerability Type**: Unlocked third-party dependencies and non-reproducible package installation **Risk Level**: Medium ### Vulnerable Code ```python def generate_requirements(config): """生成 requirements.txt""" base_deps = [ 'torch>=2.0.0', 'torchvision>=0.15.0', 'numpy>=1.24.0', 'pandas>=2.0.0', 'scikit-learn>=1.3.0', 'tqdm>=4.65.0', 'pyyaml>=6.0', ] dataset_type = config.get('dataset', {}).get('type', 'image') model_name = config.get('model', {}).get('backbone', '').lower() if dataset_type == 'text' or 'bert' in model_name: base_deps.extend([ 'transformers>=4.30.0', 'tokenizers>=0.13.0', ]) return '\n'.join(base_deps) ``` The generated README directs users to install these mutable requirements: ```markdown ### 1. 安装依赖 ```bash pip install -r requirements.txt ``` ``` The command-line completion message repeats the same instruction: ```python print(f" 2. pip install -r requirements.txt") ``` ### Technical Analysis Every generated dependency uses a lower-bound-only constraint. Consequently, the exact installed package versions and transitive dependency graph depend on what is available from the configured package index at installation time. The generated environment is therefore not reproducible and may silently consume future releases that have not been reviewed or tested with the generated project. The installation command also does not require package hashes or identify an approved package index. The audited code does not contain an explicitly malicious package name, a typosquatted dependency, or a suspicious package repository. The risk arises from mutable depend ...[truncated 1487 chars]
Remediation
## Remediation Suggestions 1. Replace lower-bound-only dependency declarations with exact, reviewed versions. 2. Produce a lock file that includes all transitive dependencies for supported Python versions and platforms. 3. Include cryptographic hashes and recommend installation with hash verification, such as: ```bash pip install --require-hashes -r requirements.lock ``` 4. Generate dependency manifests through a reproducible tool such as `pip-tools`, Poetry, or another reviewed lock-file workflow. 5. Document and enforce an approved HTTPS package index rather than relying silently on arbitrary user configuration. 6. Run dependency vulnerability and provenance scanning as part of release and generation testing. 7. Recommend installation in a dedicated, unprivileged virtual environment. Explicitly warn users not to run package installation as root or with unnecessary administrative privileges. 8. Establish a controlled dependency-update process in which new versions are tested and reviewed before lock files are refreshed.
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (12)

Tp4

High
Category
MCP Tool Poisoning
Confidence
96% confidence
Finding
The skill claims broad capability—parsing uploaded plans and generating full pipelines across PyTorch, TensorFlow, and scikit-learn—but the finding indicates the actual implementation is much narrower and incomplete. Security-relevant mismatches like this are dangerous because users and orchestrators may trust the skill with inputs, file access, or automation decisions under false assumptions, leading to unsafe execution, broken outputs, or misuse of generated code.

Lp3

Medium
Category
MCP Least Privilege
Confidence
89% confidence
Finding
The skill describes behavior that reads user-uploaded documents and generates multiple files on disk, but it does not declare any explicit tool or permission scope. This creates an authorization ambiguity: an agent may be granted broader file read/write capability than the user expects, increasing the risk of unintended file access or persistence beyond the intended workspace.

Natural-Language Policy Violations

Medium
Confidence
95% confidence
Finding
The instruction `关键代码段添加中文注释` forces a specific language in generated output. The file does not offer a language preference option or explain a region-specific requirement, so this is a natural-language locale policy violation.

Natural-Language Policy Violations

Medium
Confidence
91% confidence
Finding
The file’s natural-language docstring and user-facing messages are written only in Chinese, including the title and runtime output. This imposes a language choice without offering the user an alternative or documenting that the skill is region-specific.

Natural-Language Policy Violations

Medium
Confidence
89% confidence
Finding
This markdown file presents the skill reference and instructions only in Chinese, which can constitute a language/locale policy violation when no user opt-in or alternative language option is provided. The document does not state that it is intended solely for a Chinese-language or region-specific context.

Context-Inappropriate Capability

Medium
Confidence
91% confidence
Finding
For text datasets, the generated code calls BertTokenizer.from_pretrained('bert-base-chinese'), which typically fetches tokenizer assets from external repositories if not already cached. The manifest describes a skill that generates runnable ML pipeline code from an experiment plan, but does not justify adding network-dependent model download behavior as part of that purpose.

Natural-Language Policy Violations

Medium
Confidence
93% confidence
Finding
This Python file generates a dataset module that always uses `bert-base-chinese`, and elsewhere also generates a BERT model defaulting to the same Chinese-only checkpoint. That imposes a specific language/locale behavior on downstream use without offering opt-in, selection, or explaining that the skill is limited to Chinese-language experiments.

Natural-Language Policy Violations

Medium
Confidence
95% confidence
Finding
The generated `create_model` function defaults `model_name` to `bert-base-chinese`, which bakes in a specific language choice for text classification workflows. Under the policy, forcing a locale/language without user opt-in is a natural-language policy violation unless clearly justified as region-specific.

Context-Inappropriate Capability

Medium
Confidence
92% confidence
Finding
The generated BERT path uses BertForSequenceClassification.from_pretrained(model_name), which commonly downloads model weights and configuration from remote hubs. While code generation is within scope, embedding automatic remote model retrieval is a separate capability not stated in the manifest for a paper-to-pipeline generator.

Missing User Warnings

Low
Confidence
82% confidence
Finding
This markdown file describes producing a concrete output directory with multiple files such as `main.py`, `config.yaml`, and `README.md`, which implies filesystem writes affecting user data or workspace state. Under the markdown-file criteria for SQP-2, the description should warn users about this behavior because it can modify the user's project directory.

Natural-Language Policy Violations

Low
Confidence
95% confidence
Finding
该 markdown 文档从标题到内容均固定为中文,没有说明这是特定区域/受众限定的材料,也没有提供其他语言选项。根据语言/区域政策,强制单一语言且缺少用户选择或明确合理性说明,属于自然语言层面的政策风险。

Context-Inappropriate Capability

Low
Confidence
84% confidence
Finding
The generated image-model code sets pretrained=True by default and passes torchvision weight enums into model constructors, which can trigger downloading pretrained weights when absent locally. The skill is described as generating experiment pipeline code, not provisioning external artifacts, so this additional capability is not clearly justified by the manifest.

Static analysis

Detected: suspicious.dynamic_code_execution

Dynamic code execution detected.

Critical
Code
suspicious.dynamic_code_execution
Location
assets/templates/image_classification.py:105

Dynamic code execution detected.

Critical
Code
suspicious.dynamic_code_execution
Location
scripts/generate_pipeline.py:818