Fine-Tuning

• Outcome: Pass
• Latest audit: Sep 10, 2026 · 6:42 PM UTC
• Version: 1.0.0

Overview

This documentation-only fine-tuning skill is coherent and purpose-aligned, but users should be careful before uploading datasets or enabling provider data sharing.

Before installing or using this skill, treat all fine-tuning datasets as sensitive. Do not use the cloud upload, augmentation, evaluation, or data-sharing examples unless the data is redacted or approved for that provider, and pin package/model versions when following offline setup examples.

A.I.G

Vulnerability Patterns

• Insecure Dependencies: Introduces malicious components through unsafe dependency sources
• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review

Findings (2)

T08 · Insecure Dependencies

Warning

• Location: compliance.md:103

Finding

Unpinned Third-Party Packages and Model Artifacts

Content

View full analysis

Vulnerability Details

File Location: compliance.md:103-110
Vulnerability Type: Supply-chain integrity weakness caused by unpinned dependencies and model revisions
Risk Level: Medium

Vulnerable Code:

bash

# Pre-download everything
pip download torch transformers unsloth -d ./packages/
huggingface-cli download meta-llama/Llama-3.1-8B --local-dir ./models/

# Transfer to air-gapped system
# Install from local
pip install --no-index --find-links=./packages/ torch transformers unsloth

Technical Analysis

The documented commands download packages without exact version constraints, cryptographic hashes, or an approved package index. The Hugging Face model is also downloaded without pinning an immutable commit revision.

Consequently, the artifacts transferred into the air-gapped environment depend on whichever package versions and model revision are resolved at download time. The use of --no-index during installation only prevents network access on the destination system; it does not verify the provenance or integrity of artifacts previously placed in ./packages/.

Python packages may execute installation or import-time code. Model repositories can also contain configuration or custom modeling code, depending on how downstream loading is configured. A compromised upstream release, account, package source, or mutable model repository could therefore introduce altered content into an otherwise trusted environment.

Attack Path

1. An upstream package release, package-distribution account, dependency, or model repository is compromised or publishes a malicious revision.
2. An operator runs the documented unpinned download commands.
3. Dependency resolution retrieves the compromised or unintended latest artifacts.
4. The operator transfers those artifacts to the air-gapped system under the assumption that they are trusted.
5. The packages are installed locally, or the model is subsequent ...[truncated 932 chars]

Remediation

View remediation

Remediation Suggestions

• Pin every direct and transitive Python dependency to an exact reviewed version.
• Generate a locked requirements file containing SHA-256 hashes and install with pip install --require-hashes.
• Specify an explicitly approved package index during acquisition and disable unapproved extra indexes.
• Pin the Hugging Face model to an immutable commit using the appropriate revision option.
• Produce a signed artifact manifest containing filenames, versions, origins, licenses, and cryptographic hashes.
• Verify the manifest before transfer and again inside the air-gapped environment.
• Scan packages and model files for malware and known vulnerabilities before approval.
• Avoid loading remote custom model code unless it has been separately reviewed and pinned.
• Perform installation and training inside a non-privileged, network-restricted container or virtual environment.

T09 · Insecure Skill Coding Practices

Note

• Location: compliance.md:18

Finding

PII Scanner Retains Complete Sensitive Values in Its Report

Content

View full analysis

Vulnerability Details

File Location: compliance.md:18-46
Vulnerability Type: Plaintext duplication and potential disclosure of sensitive personal data
Risk Level: Low

Vulnerable Code:

python

def scan_for_pii(text):
    findings = []
    for pii_type, pattern in PII_PATTERNS.items():
        matches = re.finditer(pattern, text, re.IGNORECASE)
        for match in matches:
            findings.append({
                "type": pii_type,
                "value": match.group(),
                "position": match.span()
            })
    return findings

def audit_dataset(path):
    report = {"total_examples": 0, "pii_found": []}

    with open(path) as f:
        for i, line in enumerate(f, 1):
            obj = json.loads(line)
            report["total_examples"] += 1

            for msg in obj.get("messages", []):
                findings = scan_for_pii(msg.get("content", ""))
                if findings:
                    report["pii_found"].append({
                        "line": i,
                        "findings": findings
                    })

    return report

Technical Analysis

The scanner stores match.group() for every detected email address, phone number, Social Security number, IP address, and credit-card-like value. This duplicates complete sensitive values from the source dataset into the returned report.

A remediation report generally needs only the finding type, source line, character span, and possibly a masked or keyed fingerprint. Retaining the complete value expands the number of in-memory and potentially persisted copies of personal data. The risk becomes actionable when callers print, serialize, log, transmit, or attach the report to an issue-tracking or audit system.

The code does not itself persist or transmit the report, so exploitation requires downstream handling that exposes the returned data.

Attack Path

1. A JS ...[truncated 1144 chars]

Remediation

View remediation

Remediation Suggestions

• Remove the raw value field from findings by default.
• Return only the PII type, source line, character span, and a safely masked preview when operationally necessary.
• Use a keyed HMAC rather than a plain hash if findings must be correlated without retaining their original values.
• Mark reports as sensitive and prevent them from being written to ordinary application logs.
• Encrypt persisted reports and enforce least-privilege access controls.
• Define short retention periods and secure deletion procedures for audit reports.
• Add explicit caller documentation warning against printing or transmitting raw findings.
• Validate matches before remediation because the regular expressions can produce false positives, particularly for IP addresses and payment-card-like numbers.

SkillSpector

Vulnerability Patterns

• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access
• Supply Chain: Unpinned Dependencies, External Script Fetching, Obfuscated Code
• Excessive Agency: Unrestricted Tool Access, Autonomous Decision Making, Scope Creep

Findings (4)

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 91% confidence

Finding

The guidance recommends enabling provider data sharing as a cost optimization and only briefly notes a privacy tradeoff, which can normalize a risky default for users handling fine-tuning or inference data. In a fine-tuning skill, datasets often contain proprietary, regulated, or sensitive information, so understated privacy warnings can lead to accidental exposure or policy noncompliance.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 95% confidence

Finding

The synthetic data generation example sends the full training example to an external model API, which can expose sensitive prompts, user content, or proprietary data if operators reuse real examples during augmentation. In a fine-tuning/data-preparation skill, this is especially risky because datasets commonly contain confidential or regulated information, and the example provides no warning, sanitization step, or consent/compliance guidance before transmission.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 92% confidence

Finding

The document explicitly promotes a 50% discount for enabling data sharing but provides no warning that training data may contain sensitive, proprietary, regulated, or personal information. In a fine-tuning skill, this omission is materially risky because users are likely handling real datasets, so cost optimization guidance can nudge them into weakening privacy controls without informed consent.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 94% confidence

Finding

The example code transmits a local file (training.jsonl) to a third-party API for fine-tuning without any accompanying warning, consent step, or guidance to verify that the dataset is safe to export. In a fine-tuning skill, training data often contains proprietary, regulated, or personal information, so omission of an explicit data-handling warning materially increases the risk of accidental sensitive data disclosure.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.