Back to skill

Security audit

skill-evaluator-srl

Security checks for vulnerabilities and agentic risk

Overview

The skill appears to be a legitimate SRL skill-review tool, but its instructions include an unsafe shell pattern that can turn reviewed skill content into command execution.

Install only if you are comfortable with a skill evaluator that reads target skill directories, runs a local Python analyzer, and writes reports. Do not follow the documented echo '<JSON>' shell examples with untrusted skill content; use file-based JSON input and non-shell process invocation, and confirm output paths before writing reports.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Error
Location
SKILL.md:452
Finding

Shell Command Injection Through AI-Generated JSON and Unquoted Paths

Content
View full analysis
' | python3 /srl_analyzer.py > /path/to/target-skill/.srl-report.md echo '' | python3 /srl_analyzer.py --json > /path/to/target-skill/.srl-report.json ``` The same unsafe command-construction pattern also appears in `SKILL.md` at lines 421 and 424. ### Technical Analysis The documented workflow instructs the agent to place generated JSON directly inside a single-quoted shell command. That JSON contains fields derived from the audited Skill, including its description, evidence, paths, and other potentially attacker-controlled content. A single quote in the generated data terminates the shell string. Any shell syntax following that quote is then interpreted as a command rather than JSON data. JSON does not escape apostrophes because they have no special meaning in JSON strings, so ordinary JSON serialization does not prevent this issue. For example, attacker-controlled Skill content resembling the following could be incorporated into a generated JSON field: ```text '; id > /tmp/srl-command-injection-proof; # ``` If inserted into the documented command, the shell can interpret it as: ```bash echo '{"description":"'; id > /tmp/srl-command-injection-proof; # ...' ``` The `id` command would execute with the privileges of the agent running the evaluation. The script path and output path placeholders are also unquoted. If an attacker can influence the target path and that path contains whitespace or shell metacharacters, substituting it directly into the command creates an additional command-injection and argument-confusion vector. The Python implementation itself does not invoke a shell or subprocess. The vulnerability is in the Skill instructions that direct the agent to construct and execute unsafe shell c ...[truncated 1882 chars]
Remediation
View remediation
Vulnerability Patterns
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (12)

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 38)May include surrounding context.

md
- [ ] 确认目标路径存在且包含 `SKILL.md`

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

SQP-3 applies to all file types and covers language or locale policy violations. The natural-language instructions and usage guidance force a single language experience, and the file does not state that the tool is China-specific or otherwise justify the restriction, nor does it offer an opt-in language choice.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The trigger list includes generic phrases such as '检查 skill' and 'skill 质量' that can cause unintended activation during ordinary discussion or review tasks. In an agent ecosystem, overly broad routing can invoke this skill on unrelated inputs, creating incorrect tool selection, unnecessary file access, or interference with higher-priority security workflows.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
95% confidence
Finding

The skill instructs reading multiple files from a target skill and project directories, but it declares no explicit tool scope such as permissions or allowed-tools. This creates an authority/expectation mismatch: an agent may be induced to use file-read capabilities more broadly than the manifest communicates, increasing the chance of unintended data exposure during evaluation.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

L003 将触发/描述关键词写成“评估 skill、skill 评分、检查 skill、skill review、skill score”等宽泛短语,但没有限定使用场景、明确触发词列表边界或排除条件。这些表述在一般对话中也很常见,容易让技能在用户只是泛泛谈论评审或打分时被意外匹配。

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The skill directs the agent to execute a local Python script via shell to process evaluation data. Running local scripts is a materially stronger capability than passive review: if the script, path resolution, or input handling is compromised, the evaluator becomes an execution vector for arbitrary local code under the guise of scoring.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The skill instructs saving .srl-report.md and .srl-report.json into the target skill directory, which upgrades a review task into a file-modifying task. In context, this is more dangerous because the target path is derived from user-selected skills; a path selection mistake or abuse could overwrite artifacts in unintended locations and create persistence inside evaluated repositories.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

This markdown file is entirely written in Chinese from the title onward, which imposes a specific language choice in the skill documentation. The file does not indicate that language is optional, user-selectable, or justified as a region-specific artifact, so it conflicts with the stated policy on language/locale opt-in.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
98% confidence
Finding

This markdown file contains user-facing guidance exclusively in Chinese, including the title, section headings, and explanatory text. Under the policy rules, forcing a specific language without offering the user a language or locale choice is a natural-language policy violation.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

This code file contains natural-language strings and documentation entirely in Chinese, including the module docstring and later user-facing help/report text, with no indication that users can choose another language. Under the policy rule, forcing a specific language without user opt-in is a natural-language policy violation.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The report generator emits headings, labels, warnings, and recommendations in Chinese, and the schema/help printer does the same elsewhere in the file. Because these are user-facing outputs with no configurable locale selection, the skill enforces a single language rather than offering user choice.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
91% confidence
Finding

The template hard-codes bilingual Chinese/English section headings and labels throughout the document, which imposes a specific language/locale format by default. Under the policy, language constraints should either be optional for the user or clearly justified as region-specific.

Content

No source excerpt is available for this finding.