Security audit

Improvement Discriminator

Security checks for vulnerabilities and agentic risk

Overview

This is mainly a scoring skill, but it also contains under-disclosed code execution and pipeline-routing behavior that users should review before installing.

Install only if you are comfortable with a review skill that can influence an improvement pipeline and whose interface module can execute local Python skill code when used that way. Treat accept_for_execution as advisory unless a separate gate enforces policy, use mock or local-only judging for sensitive content, and do not point the real evaluator at untrusted skill files without sandboxing.

SkillSpector

Vulnerability Patterns

Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands

Findings (17)

Lp3

Medium

Category: MCP Least Privilege
Confidence: 80% confidence
Finding: The skill advertises CLI behavior that uses environment variables for API keys and reads/writes local files, but the manifest does not declare any permissions. This creates a transparency and policy-enforcement gap: callers may invoke a skill with more capability than expected, making accidental secret exposure or unsafe file access more likely.

Tp4

High

Category: MCP Tool Poisoning
Confidence: 93% confidence
Finding: The documented purpose is narrow candidate scoring, but the described behavior reportedly extends to dynamic loading/execution of Python skill modules, running hidden/frozen tests, importing external test results, and gate-adjacent decisioning. That mismatch is dangerous because operators may trust the skill in lower-risk review contexts while it actually performs code execution and broader evaluation actions with materially higher security impact.

Description-Behavior Mismatch

Medium

Confidence: 84% confidence
Finding: The manifest says the skill is not for gate decisions, yet it also says it determines whether candidates should be accepted and recommends execution actions. This ambiguity can cause users or orchestrators to route approval authority through a skill that claims not to have it, weakening control separation and review safeguards.

Intent-Code Divergence

Medium

Confidence: 83% confidence
Finding: The later documentation contradicts the earlier statement that the skill is not for gate decisions by describing acceptance and execution recommendations. In practice, contradictory guidance increases the chance of unsafe automation, where a scoring tool is treated as an authorization tool without the stricter controls expected for gating.

Description-Behavior Mismatch

High

Confidence: 81% confidence
Finding: The code computes verdicts such as 'production_ready' and 'stable', which are effectively gate or acceptance decisions despite the manifest stating this skill is not for gate decisions. That mismatch can cause downstream systems or operators to rely on this component for release decisions it was not supposed to make, increasing the chance of unsafe promotion or policy bypass.

Description-Behavior Mismatch

High

Confidence: 98% confidence
Finding: This code loads a Python file from a provided filesystem path and executes it via importlib and a discovered function, which enables arbitrary code execution if the path is attacker-controlled or points to untrusted content. In a judging/scoring skill, that capability is far beyond the declared purpose and materially increases the attack surface.

Context-Inappropriate Capability

High

Confidence: 99% confidence
Finding: The module uses importlib.util.spec_from_file_location(), module_from_spec(), and exec_module() to import and run arbitrary Python code from disk. That is a classic arbitrary code execution pattern and is especially dangerous here because the skill's role is scoring/judging, not trusted code execution.

Description-Behavior Mismatch

High

Confidence: 95% confidence
Finding: This module adds broad external regression import, score aggregation, and report-export behavior that is materially outside the skill’s declared purpose of blind-review scoring and LLM semantic evaluation. In an agent setting, capability drift is dangerous because it silently expands what the skill can influence, including ingesting untrusted external artifacts and affecting downstream scoring/reporting logic beyond the manifest-approved scope.

Context-Inappropriate Capability

High

Confidence: 98% confidence
Finding: The merge_into_score function directly combines external regression results with a base score, creating an implicit gating/evaluation mechanism despite the manifest explicitly stating the skill is not for gate decisions. Because the imported results can come from external files and adapters, this gives out-of-scope data a direct path to alter final acceptance-relevant scoring.

Intent-Code Divergence

Medium

Confidence: 90% confidence
Finding: The file-level design and documentation describe an external regression-test integration hook, which conflicts with the declared skill intent. This mismatch increases the chance that operators, reviewers, or orchestrators will trust and invoke capabilities the skill was not authorized to have, weakening governance and making the out-of-scope scoring path harder to detect.

Description-Behavior Mismatch

High

Confidence: 94% confidence
Finding: This module exposes score-merging and report-generation logic that can directly influence acceptance outcomes, which conflicts with the skill metadata stating it should not be used for gate decisions. In practice, downstream callers can use merge_scores() and review decisions as release gating signals, creating a policy-bypass path where a supposedly advisory skill becomes an approval authority.

Intent-Code Divergence

Medium

Confidence: 98% confidence
Finding: The code describes the field as a digital signature for integrity, but _compute_signature() is only a truncated SHA-256 over predictable content with no secret key or asymmetric signing. Anyone who can modify a receipt can recompute this value, so consumers may falsely trust tampered review artifacts as authenticated and intact.

Description-Behavior Mismatch

High

Confidence: 95% confidence
Finding: This code is making substantive gate decisions by returning accept_for_execution, hold, or reject based on risk and scoring logic, even though the skill metadata says it should not be used for gate decisions. In an agent pipeline, role confusion like this is dangerous because downstream components may treat these outputs as authoritative and execute changes without the intended independent gate step.

Description-Behavior Mismatch

High

Confidence: 98% confidence
Finding: The script not only scores candidates but also sets next_step to execute_candidate and next_owner to executor, which actively routes artifacts toward execution. In the context of an agent skill, this is a control-flow escalation: a component advertised as a discriminator becomes an orchestrator that can advance potentially unsafe candidates into action.

Intent-Code Divergence

Medium

Confidence: 84% confidence
Finding: The documentation and CLI framing present this as a critic/scorer, but the implementation performs recommendations and execution routing. This mismatch is security-relevant because operators and downstream automation may trust the documented scope and grant this tool permissions or integration points inappropriate for a component that can effectively influence execution decisions.

Missing User Warnings

Medium

Confidence: 87% confidence
Finding: The component executes local Python modules without clear user-facing disclosure or consent at the point of action. Lack of disclosure does not create the code-execution primitive, but it materially increases risk because users may invoke a 'judge' skill expecting analysis, not host-side execution of arbitrary code.

Missing User Warnings

Medium

Confidence: 84% confidence
Finding: The Claude/OpenAI backends send the constructed prompt to external APIs, and that prompt includes slices of `target_content` plus candidate change content. In a skill-evaluation context, those fields may contain proprietary instructions, internal prompts, or sensitive material, and this code provides no consent gate, redaction step, or explicit disclosure at the transmission point. The issue is more concerning here because the component is specifically designed to judge skill content, so exfiltration of prompt/skill text is inherent to normal operation unless controls are added.

Static analysis

No suspicious patterns detected.