LLM Regression Monitor

• Outcome: Pass
• Latest audit: Sep 2, 2026 · 4:33 AM UTC
• Version: 1.1.0

Overview

This skill performs disclosed LLM regression monitoring with local reports and user-configured alerts, with some privacy and dependency hygiene caveats.

Install only in projects where storing LLM outputs and sending failure summaries to configured alert channels is acceptable. Avoid putting secrets or sensitive customer data in test prompts or outputs, add generated files to .gitignore, and pin dependency versions before using this in production.

SkillSpector

Vulnerability Patterns

• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Supply Chain: Unpinned Dependencies, External Script Fetching, Obfuscated Code
• MCP Least Privilege: Underdeclared Capability, Wildcard Permission, Missing Permission Declaration
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access

Findings (8)

Lp3

Medium

• Category: MCP Least Privilege
• Confidence: 89% confidence
• Finding: The skill instructs the agent to use shell commands, network access, environment variables, and file writes, but it does not declare permissions or otherwise surface those capabilities as part of an explicit trust boundary. That creates a real security issue because users or orchestrators may invoke the skill without understanding it can access secrets, persist data, and send data off-host via webhooks or email.

Missing User Warnings

Medium

• Confidence: 86% confidence
• Finding: The skill captures baseline LLM outputs to disk and forwards regression details through Slack, Discord, WhatsApp, or email, but it does not prominently warn that prompts, model outputs, and possibly sensitive production-like content may be stored and exfiltrated to third-party services. In a monitoring workflow, users may be encouraged to test real application prompts, which increases the chance of persisting or transmitting confidential data.

Unpinned Dependencies

Low

• Category: Supply Chain

Content

llm-behave[semantic]
pyyaml
requests
python-dotenv

• Confidence: 98% confidence
• Finding: The dependency pyyaml is declared without a version pin, which makes builds non-reproducible and can cause the skill to pick up newer package versions unexpectedly, including insecure or breaking releases. In a monitoring skill that may run on a schedule, this increases supply-chain and reliability risk over time.

Unpinned Dependencies

Low

• Category: Supply Chain

Content

llm-behave[semantic]
pyyaml
requests
python-dotenv

• Confidence: 98% confidence
• Finding: The requests package is unpinned, so installations may resolve to different versions over time depending on when and where the skill is deployed. This can silently introduce vulnerable releases or behavior changes, which is especially problematic for an automation-oriented skill that may send alerts and make outbound network calls.

Unpinned Dependencies

Low

• Category: Supply Chain

Content

llm-behave[semantic]
pyyaml
requests
python-dotenv

• Confidence: 97% confidence
• Finding: python-dotenv is included without version constraints, which exposes the environment to unpredictable dependency resolution and possible inclusion of newly disclosed vulnerable versions. Since dotenv tooling often interacts with secrets and configuration, unpinned upgrades can have outsized operational and security consequences.

Known Vulnerable Dependency: pyyaml — 8 advisory(ies): CVE-2019-20477 (Deserialization of Untrusted Data in PyYAML); CVE-2020-1747 (Improper Input Validation in PyYAML); CVE-2020-14343 (Improper Input Validation in PyYAML) +5 more

Critical

• Category: Supply Chain
• Confidence: 95% confidence
• Finding: pyyaml has multiple serious advisories, including unsafe deserialization and input validation issues in certain versions. Because the requirement is unpinned, the project does not demonstrate that it avoids affected releases, and if the skill parses YAML from untrusted sources, exploitation could lead to code execution or other severe compromise.

Known Vulnerable Dependency: requests — 10 advisory(ies): CVE-2014-1830 (Exposure of Sensitive Information to an Unauthorized Actor in Requests); CVE-2024-47081 (Requests vulnerable to .netrc credentials leak via malicious URLs); CVE-2024-35195 (Requests `Session` object does not verify requests after making first request wi) +7 more

High

• Category: Supply Chain
• Confidence: 93% confidence
• Finding: requests has several known advisories affecting some versions, including credential leakage and TLS/session verification issues. In this skill context, outbound HTTP is likely used for monitoring or alerting integrations, so a vulnerable version could expose secrets, mis-handle authentication, or weaken transport security.

Known Vulnerable Dependency: python-dotenv — 2 advisory(ies): CVE-2026-28684 (python-dotenv: Symlink following in set_key allows arbitrary file overwrite via ); CVE-2026-28684 (python-dotenv reads key-value pairs from a .env file and can set them as environ)

Medium

• Category: Supply Chain
• Confidence: 88% confidence
• Finding: python-dotenv is flagged with advisories related to unsafe file handling, including symlink-following behavior that may enable arbitrary file overwrite in affected versions. This skill likely relies on environment-based configuration for tokens and alert credentials, so vulnerable dotenv handling can increase the risk of local configuration tampering or secret-management abuse.

VirusTotal

53/53 vendors flagged this skill as clean.

Static analysis

No suspicious patterns detected.