Security audit

Improvement Discriminator

Security checks across malware telemetry and agentic risk

Overview

This scoring skill is mostly coherent, but it includes under-disclosed local Python code execution and executor handoff behavior that users should review before installing.

Install only if you intend to use it in a controlled improvement pipeline. Use mock LLM judging for sensitive skills, verify any Claude/OpenAI account and proxy settings before non-mock runs, and do not run the real-skill evaluator on untrusted Python files unless it is sandboxed. Treat accept_for_execution and next_owner=executor as advisory until a separate gate explicitly approves execution.

SkillSpector

By NVIDIA

Vulnerability Patterns

Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access

Findings (10)

Intent-Code Divergence

Medium

Confidence: 87% confidence
Finding: The docs say the skill does not make gate decisions, yet its output includes direct accept/reject decisions. This semantic inconsistency can cause downstream systems or operators to treat scoring output as authorization to approve or reject candidates, bypassing the intended gate layer. In security-sensitive automation, ambiguous decision authority is a control-failure that can lead to unsafe or unreviewed changes being accepted.

Description-Behavior Mismatch

Medium

Confidence: 89% confidence
Finding: The scope statement says the skill is not for gate decisions, but the description simultaneously says it judges whether a candidate should be accepted. That contradiction can mislead automated routing and human reviewers about what authority this skill has, increasing the risk of accidental policy bypass or incorrect pipeline wiring. Context makes this more dangerous because the skill sits in an orchestrated improvement pipeline where role confusion can propagate to later stages.

Description-Behavior Mismatch

High

Confidence: 95% confidence
Finding: This engine goes beyond passive scoring/judging and includes runtime execution of skill code plus writing artifacts to disk. In the context of a discriminator skill meant for blind review and semantic evaluation, that expanded capability creates unnecessary attack surface and enables arbitrary code execution and filesystem side effects if pointed at untrusted inputs.

Context-Inappropriate Capability

High

Confidence: 99% confidence
Finding: The code dynamically imports a Python file from a caller-provided path and immediately executes it via exec_module(), then invokes discovered functions. This is direct arbitrary code execution: a malicious skill file can run attacker-controlled code at import time or during function execution, leading to full compromise of the process, data exposure, or destructive actions.

Context-Inappropriate Capability

Medium

Confidence: 91% confidence
Finding: Modifying sys.path to load code from a sibling skill enables cross-skill trust bypass and can cause unexpected or attacker-influenced modules to be imported. In a plugin-like environment, this weakens isolation boundaries and increases the chance of dependency confusion, malicious replacement, or execution of code outside the declared scope of this skill.

Description-Behavior Mismatch

High

Confidence: 95% confidence
Finding: This code does more than discriminate or score candidates: it produces execution-oriented recommendations including "accept_for_execution", "hold", and "reject". In this skill context, that is dangerous because downstream systems or operators may treat the output as an authorization signal, bypassing the separate gate skill the manifest says should own such decisions.

Description-Behavior Mismatch

High

Confidence: 97% confidence
Finding: The script explicitly sets "next_step" to "execute_candidate" and "next_owner" to "executor", which operationally advances artifacts into execution. Given the skill metadata says it is not for gate decisions, this role confusion can cause unauthorized or insufficiently reviewed changes to be executed simply because they were scored here.

Missing User Warnings

Medium

Confidence: 86% confidence
Finding: The engine loads and executes code from a supplied path without any explicit warning, consent, or trust verification. Given that this capability is dangerous and not aligned with the expected blind-review role, the lack of a user-facing confirmation increases the likelihood of accidental execution of malicious code and makes exploitation easier through social engineering or misconfiguration.

Missing User Warnings

Medium

Confidence: 92% confidence
Finding: The code sends `target_content` and proposed skill changes to third-party LLM APIs via Anthropic/OpenAI without any explicit consent, disclosure, redaction, or policy gate. In a skill-improvement workflow, that content may include proprietary prompts, internal instructions, secrets accidentally embedded in skills, or other sensitive data, so silent transmission creates a real confidentiality and compliance risk even if the functionality is intentional.

Missing User Warnings

Medium

Confidence: 86% confidence
Finding: When LLM judging is enabled, the code passes candidate data and target skill content into a backend abstraction that may be backed by external providers. In a skill-review pipeline, those artifacts can contain proprietary instructions, prompts, internal paths, or sensitive operational context, so undisclosed transmission creates confidentiality and compliance risk.

VirusTotal

66/66 vendors flagged this skill as clean.

View on VirusTotal

Static analysis

Detected: suspicious.dynamic_code_execution, suspicious.exposed_secret_literal

Dynamic code execution detected.

Critical

Code: suspicious.dynamic_code_execution
Location: interfaces/critic_engine.py:379

Dynamic code execution detected.

Critical

Code: suspicious.dynamic_code_execution
Location: scripts/score.py:402

File appears to expose a hardcoded API secret or token.

Critical

Code: suspicious.exposed_secret_literal
Location: interfaces/critic_engine.py:685

File appears to expose a hardcoded API secret or token.

Critical

Code: suspicious.exposed_secret_literal
Location: tests/test_p1_integration.py:115

File appears to expose a hardcoded API secret or token.

Critical

Code: suspicious.exposed_secret_literal
Location: tests/test_p2a_integration.py:79