ExpertPack Eval

• Outcome: Pass
• Latest audit: May 29, 2026 · 8:22 PM UTC
• Version: 1.1.0

Overview

This skill is an evaluation tool that performs the advertised OpenRouter and user-endpoint calls, but users should treat pack and eval contents as data that may leave their machine.

Install only if you are comfortable sending pack propositions, eval questions, expected answers, required facts, and agent responses to OpenRouter and any endpoint you configure. Use a limited OpenRouter key, review eval data for secrets before running, and treat automated judge scores as advisory.

SkillSpector

By NVIDIA

Vulnerability Patterns

• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• MCP Least Privilege: Underdeclared Capability, Wildcard Permission, Missing Permission Declaration
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access
• Supply Chain: Unpinned Dependencies, External Script Fetching, Obfuscated Code

Findings (7)

Lp3

Medium

• Category: MCP Least Privilege
• Confidence: 93% confidence
• Finding: The skill explicitly instructs users to run Python scripts that access environment variables, read/write local files, and make external network calls, yet it declares no corresponding permissions. This creates a transparency and consent problem: users or the hosting platform may invoke the skill without understanding that it can exfiltrate API keys, send pack contents to third-party services, or modify local artifacts such as manifests and eval outputs.

Missing User Warnings

Medium

• Confidence: 96% confidence
• Finding: The script sends raw proposition text to OpenRouter to generate targeted questions, which can expose proprietary, confidential, or otherwise sensitive pack content to a third-party service. In this skill's context, the whole point is evaluating potentially unique 'esoteric knowledge,' so transmitting those facts externally is especially risky because it can leak the very high-value content being measured.

Missing User Warnings

Medium

• Confidence: 95% confidence
• Finding: The judge step transmits the ground-truth proposition and the probed model's response to another external LLM, creating a second disclosure path for sensitive pack contents and derived outputs. Because this evaluator is designed for high-value expertpack material, sending both truth data and responses to a remote judge increases the chance of unintended data exposure and widens third-party access.

Missing User Warnings

Medium

• Confidence: 94% confidence
• Finding: The script intentionally sends eval questions to a user-specified agent endpoint and then forwards the full agent response, expected answer, required facts, and anti-hallucination checks to OpenRouter for LLM-based scoring. This creates a real data disclosure risk if eval sets or responses contain proprietary, sensitive, or regulated information, and the script provides no explicit consent gate, redaction step, or strong warning before transmission.

External Transmission

Medium

• Category: Data Exfiltration

Content

}
    
    try:
        resp = requests.post(OPENROUTER_BASE, headers=headers, json=payload, timeout=15)
        resp.raise_for_status()
        question = resp.json().get("choices", [{}])[0].get("message", {}).get("content", "").strip()
        if question:

• Confidence: 93% confidence
• Finding: This outbound request is the concrete mechanism by which proposition-derived content leaves the local environment and is sent to OpenRouter. While network access is expected for this tool, it still constitutes a real data exfiltration channel if users run it on non-public or proprietary packs without understanding what is transmitted.

External Transmission

Medium

• Category: Data Exfiltration

Content

for attempt in range(max_retries):
        try:
            resp = requests.post(OPENROUTER_BASE, headers=headers, json=payload, timeout=30)
            
            if resp.status_code == 429:
                retry_after = int(resp.headers.get("Retry-After", 5))

• Confidence: 88% confidence
• Finding: This request sends probe questions to an external model and receives answers, which is inherent to the evaluator's function but still an external transmission path. The danger is moderate because the questions are derived from propositions and may still reveal sensitive topics or partial proprietary details even if the full answer is not included.

External Transmission

Medium

• Category: Data Exfiltration

Content

}
    
    try:
        resp = requests.post(OPENROUTER_BASE, headers=headers, json=payload, timeout=15)
        resp.raise_for_status()
        
        judge_answer = resp.json().get("choices", [{}])[0].get("message", {}).get("content", "").strip().upper()

• Confidence: 94% confidence
• Finding: This outbound judge request sends ground truth plus model response to a third-party API, exposing sensitive source content more directly than the probe step. In this evaluator, that means the external service may see exact expertpack facts that could be confidential or commercially valuable.

VirusTotal

65/65 vendors flagged this skill as clean.

Static analysis

No suspicious patterns detected.