Back to skill

Security audit

eval-integrity

Security checks for vulnerabilities and agentic risk

Overview

This skill performs a disclosed, read-only benchmark audit; its GitHub PR lookup broadens scope slightly but remains non-mutating and purpose-aligned.

Install this if you want a rigorous read-only audit of a benchmark repository. Be aware that the workflow may query GitHub PR metadata with your local gh authentication; avoid running it in environments where even read-only PR metadata access is not acceptable.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Vulnerability Patterns
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Findings (3)

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The skill is presented as a read-only local repo audit, but it mandates querying GitHub PR state via gh pr list, which expands scope beyond the target repository. This is a capability mismatch: it can disclose or depend on remote metadata and network access not necessary for many local audits, though it does not itself execute code or modify state.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
84% confidence
Finding

Requiring GitHub PR enumeration as a standing, non-optional step gives the skill broader access than its core benchmark-integrity function requires. In environments with sensitive remotes, private repository metadata, or restricted network policies, this unnecessary remote lookup can leak contextual information or violate least-privilege expectations.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Low
Category
Not specified by scanner
Confidence
72% confidence
Finding

The documentation presents the skill as a read-only benchmark audit focused on the target repository, which suggests repo-contained inspection. Later instructions require checking open PRs via gh pr list, creating a mild contradiction between the narrow read-only repo-audit framing and actual behavior that reaches into external repository hosting state.

Content

No source excerpt is available for this finding.