ClawHub
Back to skill

Security audit

Llm As Judge

Security checks for vulnerabilities and agentic risk

Overview

This skill is a coherent LLM evaluation guide that discloses its expected use of external judge APIs, but users should treat evaluated content as potentially shared with those providers.

Install only if you are comfortable sending sampled evaluation artifacts to Anthropic/OpenAI judge models. Redact sensitive prompts, customer data, ground truth, source text, and model outputs before judging, and be careful about storing raw judge responses in logs.

A.I.G

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs

SkillSpector

Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
Findings (1)

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The skill explicitly states it makes outbound calls to Anthropic and OpenAI judge models, but it does not clearly warn users that candidate outputs, ground truth, source text, and possibly other evaluation artifacts may be sent to third-party providers. In an evaluation workflow, those inputs often contain proprietary prompts, model outputs, customer data, or sensitive internal content, so missing disclosure can lead to unintentional data exfiltration and compliance/privacy violations.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.