Back to skill

Security audit

Skill Test

Security checks for vulnerabilities and agentic risk

Overview

This skill is a coherent local evaluator for agent skills, with disclosed file-reading and optional eval scaffold writing that fit its purpose.

Install this if you want local QA guidance and a static evaluator for agent skills. Review the target skill before running any runtime tests, use isolation for unfamiliar or side-effectful skills, and avoid --scaffold unless replacing or creating eval.yaml is acceptable.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Vulnerability Patterns
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (17)

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 3)May include surrounding context.

md
Hub, skills.sh, and similar directories. Includes the bundled static evaluator `scripts/eval_skill.py` plus guidance for optional deterministic or LLM-assisted

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 28)May include surrounding context.

md
Hub, skills.sh, and similar directories. Includes the bundled static evaluator `scripts/eval_skill.py` plus guidance for optional deterministic or LLM-assisted

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 126)May include surrounding context.

md
Hub, skills.sh, and similar directories. Includes the bundled static evaluator `scripts/eval_skill.py` plus guidance for optional deterministic or LLM-assisted

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 18)May include surrounding context.

md
Always start by reading the target `SKILL.md`, then run:

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 50)May include surrounding context.

md
Always start by reading the target `SKILL.md`, then run:

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 103)May include surrounding context.

md
Always start by reading the target `SKILL.md`, then run:

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 138)May include surrounding context.

md
Always start by reading the target `SKILL.md`, then run:

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 146)May include surrounding context.

md
Always start by reading the target `SKILL.md`, then run:

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 164)May include surrounding context.

md
Always start by reading the target `SKILL.md`, then run:

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 252)May include surrounding context.

md
Always start by reading the target `SKILL.md`, then run:

Self-Modification

High
Category
Rogue Agent
Confidence
85% confidence
Finding

Skill modifies its own code, configuration, or behavior at runtime. Self-modification enables an agent to escalate privileges, disable safety constraints, or install persistent backdoors.

Content

Scanner excerpt · eval.yaml (reported line 22)May include surrounding context.

yaml
- id: negative-trigger-seo-only
    prompt: |
      Rewrite this skill description so it ranks better on skills.sh, but do not run any quality evaluation.
    expect:
      should_trigger: false

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
70% confidence
Finding

Without declared permissions the skill's intent is opaque and cannot be validated.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
87% confidence
Finding

The description says to use this skill when you need to "test a skill, write evals, benchmark quality, catch regressions, audit trigger accuracy, compare versions, or decide whether a skill is ready to publish." Several phrases like "write evals," "benchmark quality," and "compare versions" are broad and could overlap with ordinary software-review requests unless the target being evaluated is explicitly a skill.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The trigger case uses broad language about being 'ready to publish,' 'release blockers,' and 'regression risks,' which can match ordinary review or project-readiness discussions rather than a clear request to invoke this specific skill. That ambiguity increases the chance of unintended triggering, causing the agent to activate the skill in contexts where the user did not actually request a formal skill evaluation workflow.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
83% confidence
Finding

This markdown file includes example positive trigger categories such as 'Task description: Set up X with Y for Z' and a broad natural-language request about scaffolding a demo. These examples are generic enough to match many ordinary user requests, and the document does not pair them with concrete scope constraints for when a skill should or should not activate.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

In scaffold mode, the code unconditionally writes to skill_dir / "eval.yaml". Although it logs after the fact, there is no prior confirmation or overwrite warning in code comments/docstrings at the write site, so a user may not realize an existing eval.yaml will be replaced.

Content

No source excerpt is available for this finding.

Vague Triggers

Low
Category
Not specified by scanner
Confidence
79% confidence
Finding

The prompt "Run the skill-test grader against this skill folder and show me the report" is reasonably specific, but the surrounding examples also normalize generic evaluation language across the document without consistently requiring explicit mention of agent skills. In a markdown skill description, that can make the trigger boundary less crisp for routing systems.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.