Github Experiment Accuracy

• Outcome: Review
• Latest audit: Sep 16, 2026 · 6:50 AM UTC
• Version: 1.0.0

Overview

This skill has a coherent accuracy-testing purpose, but it tells agents to load models from arbitrary GitHub repositories using unsafe PyTorch deserialization, which can run untrusted code.

Only use this skill in a disposable, sandboxed environment with no secrets, credentials, or sensitive files available. Do not run it against arbitrary repositories or model files unless the repository, exact commit, and model artifact are trusted and verified; prefer safe model formats or weights-only loading.

A.I.G

Vulnerability Patterns

• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Embedded Malicious Code: Ships malicious scripts inside the skill and executes them locally
• Unauthorized Access and Privilege Escalation: Obtains permissions beyond the task's legitimate needs

Findings (1)

T03 · Remote Payload Retrieval and Execution

Error

• Location: SKILL.md:23

Finding

Arbitrary Code Execution Through Untrusted PyTorch Model Deserialization

Content

View full analysis

Vulnerability Details

File Location: SKILL.md, lines 23-53
Vulnerability Type: Unsafe deserialization of a remotely retrieved model
Risk Level: Critical

Vulnerable Code

bash

# Prefer git clone
git clone {github_url} {project_dir} --depth 1

powershell

Invoke-WebRequest -Uri "{github_url}/archive/refs/heads/main.zip" -OutFile "outputs/repo.zip"
Expand-Archive -Path "outputs/repo.zip" -DestinationPath "outputs/projects"

python

# Find the model file
model_file = glob.glob(f"{project_dir}/**/model.pt", recursive=True)[0]

# Load the model
net = torch.load(model_file, map_location='cpu', weights_only=False)
net.eval()

The same unsafe loading behavior is reiterated in SKILL.md, lines 126-128, and references/workflow.md, lines 25-27:

python

net = load(model_file, map_location='cpu', weights_only=False)
net.eval()

text

1. Find model files (model.pt, *.pth)
2. torch.load(weights_only=False)
3. net.eval()

Technical Analysis

The workflow accepts a user-supplied GitHub repository URL, retrieves its contents, recursively locates a model.pt file, and loads that file using torch.load with weights_only=False.

PyTorch model files loaded in this mode can use Python pickle-based object deserialization. Pickle is not a safe format for untrusted input because object reconstruction can invoke attacker-controlled callables. Consequently, a malicious model can execute code during torch.load; execution occurs before net.eval() and does not require the model to be valid or inference to begin.

The exposure is compounded by the absence of repository commit pinning, model checksum verification, publisher authentication, or an approved-model allowlist. Recursive selection of the first matching model.pt also allows the remote repository to control which artifact is loaded.

This creates a direct remote payload retrieval and execution path: t ...[truncated 2017 chars]

Remediation

View remediation

Remediation Suggestions

1. Prohibit unrestricted deserialization

   • Remove all instructions recommending weights_only=False.
   • Load only state dictionaries with torch.load(..., weights_only=True) where supported.
   • Do not fall back automatically to unrestricted loading when safe loading fails.

2. Prefer non-executable model formats

   • Require SafeTensors or another format that cannot embed pickle reconstruction logic.
   • Reconstruct the model architecture from reviewed local code and load only validated tensor weights.

3. Authenticate remote inputs

   • Restrict repositories to trusted owners or an explicit allowlist.
   • Pin each repository to a reviewed immutable commit hash rather than a mutable branch.
   • Require an approved cryptographic checksum for every model artifact and verify it before loading.

4. Validate model selection

   • Do not select the first recursive model.pt match.
   • Require an explicit, expected model path.
   • Reject symbolic links, path traversal, unexpected file types, and model files outside the canonical repository directory.

5. Isolate unavoidable legacy loading

   • If a legacy pickle model must be inspected, deserialize it only in a disposable sandbox or virtual machine.
   • Disable outbound network access and remove credentials and secrets from the environment.
   • Mount the host filesystem read-only or expose only a temporary working directory.
   • Run as a dedicated unprivileged user with strict CPU, memory, process, and execution limits.
   • Destroy the sandbox after processing.

6. Fail securely

   • Treat safe-loader failures as validation failures.
   • Record the repository commit and verified model digest in the report.
   • Never suggest weights_only=False as routine troubleshooting guidance.

SkillSpector

Vulnerability Patterns

• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Excessive Agency: Unrestricted Tool Access, Autonomous Decision Making, Scope Creep
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access
• Supply Chain: Unpinned Dependencies, External Script Fetching, Obfuscated Code

Findings (6)

Context-Inappropriate Capability

High

• Category: Not specified by scanner
• Confidence: 99% confidence

Finding

The skill instructs loading a PyTorch model taken from an arbitrary GitHub repository with torch.load(..., weights_only=False). PyTorch pickle-based deserialization can execute attacker-controlled code during loading, so simply evaluating an untrusted repo can lead to arbitrary code execution on the analyst's machine.

Content

No source excerpt is available for this finding.

Missing User Warnings

High

• Category: Not specified by scanner
• Confidence: 99% confidence

Finding

The document recommends torch.load(weights_only=False) for model loading. PyTorch model deserialization can invoke Python pickle behavior, and loading an untrusted .pt/.pth file may lead to arbitrary code execution; the skill context makes this especially dangerous because it processes models from arbitrary GitHub repositories.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 88% confidence

Finding

The workflow directs cloning/downloading arbitrary repositories, copying local user-supplied files into them, and writing reports to the filesystem without any explicit warning, consent boundary, or isolation guidance. In this context, the skill is specifically designed to handle untrusted third-party code and local data, so omitting safety constraints materially increases the chance of unsafe execution, data exposure, or unintended host modification.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 93% confidence

Finding

The workflow explicitly instructs users to clone or download arbitrary GitHub repositories and place external data into the project without any warning, isolation guidance, or trust boundary discussion. In this skill’s context, the natural next step is to run project code against supplied data, so treating untrusted repositories and inputs as routine significantly increases the chance of executing malicious code or mishandling sensitive data.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low

• Category: Not specified by scanner
• Confidence: 95% confidence

Finding

The name and operational description instruct the skill in Chinese and do not indicate that users may choose another language. Under the stated policy, forcing a specific language without opt-in is a natural-language policy violation unless the locale restriction is clearly justified.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low

• Category: Not specified by scanner
• Confidence: 91% confidence

Finding

All user-facing instructional content in this file is presented only in Chinese, and there is no indication that the skill is region-specific or that users can opt into this locale. Per the policy rule, forcing a specific language without user opt-in is a natural-language policy concern.

Content

No source excerpt is available for this finding.