Security audit
judge-audit
Security checks for vulnerabilities and agentic risk
Overview
The skill provides proportionate instructions for testing AI judges, with disclosed API use and no hidden execution or persistence.
Start with MockJudge and synthetic cases. Hosted audits send cases to the selected provider and can incur charges. Use an isolated environment and a reviewed dependency version: [upstream documentation](https://github.com/ianalloway/juryrig) reports that 0.3.1 fixes automatic OpenAI-key forwarding to custom endpoints. Use endpoint-specific credentials for third-party services. The external dependency was not fully audited.
Vulnerability Patterns
- Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
- Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
- Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
- Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
- Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Vulnerability Patterns
- Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
- Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
- Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
- Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
- Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
