LLM Regression Monitor

• Outcome: Pass
• Latest audit: May 29, 2026 at 8:31 PM UTC
• Version:
  1.0.2

Overview

This skill does what it says by monitoring LLM outputs and sending regression alerts, but users should treat API keys, prompts, outputs, reports, and alerts as sensitive.

Install only in a trusted project, prefer a virtual environment, keep .env private and out of version control, use least-privilege API keys, avoid sensitive prompts unless approved for the chosen provider, keep Slack/WhatsApp alerts to private destinations, and add generated baseline/report/log files to .gitignore.

SkillSpector

By NVIDIA

Vulnerability Patterns

• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access
• Supply Chain: Unpinned Dependencies, External Script Fetching, Obfuscated Code
• Excessive Agency: Unrestricted Tool Access, Autonomous Decision Making, Scope Creep

Findings (4)

Missing User Warnings

Medium

• Confidence: 95% confidence
• Finding: The documentation tells users to export live API keys and place them in a project-local .env file, but provides no warning about credential sensitivity, file permissions, accidental commits, shell history, or secret rotation. That omission can lead to credential exposure through source control, shared workspaces, logs, or misconfigured environments. Because this skill is meant to run on schedules and in project roots, the risk of secrets lingering in broadly accessible locations is higher than in a one-off command.

Missing User Warnings

Medium

• Confidence: 96% confidence
• Finding: The skill supports sending alerts via WhatsApp and Slack but does not warn that prompt content, model outputs, failure details, or other monitoring metadata may be transmitted to third-party services. This can cause unintentional disclosure of sensitive business data, user inputs, or regulated content outside the primary system boundary. The skill context makes this more dangerous because regression monitoring often captures representative prompts and responses that may themselves contain confidential or production-like data.

Missing User Warnings

Medium

• Confidence: 79% confidence
• Finding: The document instructs users to send prompts to third-party LLM providers but does not warn that prompts may contain sensitive business or personal data and will be transmitted off-system. In a monitoring skill, this omission can cause users to unknowingly export confidential baseline prompts, test cases, or production-like inputs to external vendors.

Missing User Warnings

Medium

• Confidence: 87% confidence
• Finding: The script captures model outputs and persists them as baseline files, but it does not warn users that prompts and outputs may contain sensitive or regulated data. In an LLM regression-monitoring context, those saved baselines can accumulate secrets, PII, or proprietary content on disk, increasing exposure through local compromise, backups, or accidental commits.

VirusTotal

62/62 vendors flagged this skill as clean.