Vague Triggers
Medium
- Category
- Not specified by scanner
- Confidence
- 95% confidence
- Finding
该文件是 markdown，适用 SQP-1。L06 中的触发示例如“给我造个测试集”与“评测用例怎么设计”在日常开发交流中较常见，且未给出明确排除条件或负例，容易与一般性讨论场景重叠，导致不必要的技能调用。
- Content
Security audit
eval-bench-builder
Security checks for vulnerabilities and agentic risk
Overview
This skill coherently builds reproducible evaluation benchmark files and does not show hidden network access, credential handling, persistence, or destructive behavior.
Install this if you want an eval benchmark generator. Before running it, choose the input spec and output JSONL path explicitly, review generated cases before using them in regression tests, and treat ambiguous evaluation questions as discussion unless you actually want files generated.
Vulnerability Patterns
- Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
- Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
- Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
- Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
- Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Vulnerability Patterns
- Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
- Taint TrackingDirect Taint Flow, Variable-Mediated Taint Flow, Credential Exfiltration Chain
- MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
- Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
- Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (15)
Natural-Language Policy Violations
Medium
- Category
- Not specified by scanner
- Confidence
- 90% confidence
- Finding
The 'License & attribution' section is presented primarily in Chinese, including critical licensing and usage constraints such as rights reservations and license caveats. Because this file otherwise includes English content but does not provide an equivalent English version of these policy-critical instructions, it imposes a language constraint without clear user opt-in or justification.
- Content
Vague Triggers
Medium
- Category
- Not specified by scanner
- Confidence
- 93% confidence
- Finding
The trigger phrases are broad enough to match common requests about evaluating AI or building test sets, which can cause unintended invocation of the skill in situations where the user did not explicitly request benchmark generation. In agent systems, overbroad activation increases the chance of surprising file creation, misleading automation, or unintended workflow transitions.
- Content
Vague Triggers
Medium
- Category
- Not specified by scanner
- Confidence
- 91% confidence
- Finding
The phrase '收到「造个评测」类指令' is ambiguous and lacks scope boundaries, making it easier for the skill to trigger on loosely related requests. In an automated assistant environment, ambiguous routing rules can cause tools or generation flows to run when a user only wanted advice or discussion.
- Content
Vague Triggers
Medium
- Category
- Not specified by scanner
- Confidence
- 88% confidence
- Finding
The English usage guidance describes keyword-based activation in broad terms without concrete limiting conditions, which can lead to over-triggering. This is dangerous in multi-skill environments because generic evaluation-related language is common and may route users into an artifact-generating workflow unexpectedly.
- Content
Missing User Warnings
Medium
- Category
- Not specified by scanner
- Confidence
- 86% confidence
- Finding
The workflow says the skill may emit benchmark files or code snippets but does not clearly warn about file creation or workspace modification effects. Users may unknowingly permit persistent changes, which is especially risky in environments where generated files can later be executed or committed.
- Content
Missing User Warnings
Medium
- Category
- Not specified by scanner
- Confidence
- 87% confidence
- Finding
The Chinese workflow mirrors the same issue by describing benchmark file output without notifying users about workspace side effects. In practice, undisclosed file generation can cause confusion, accidental overwrites, or downstream execution of generated artifacts.
- Content
Vague Triggers
Medium
- Category
- Not specified by scanner
- Confidence
- 91% confidence
- Finding
The trigger text is broad enough to activate on generic requests about testing, benchmarks, or evaluating AI, which can cause the agent to invoke this skill outside its intended narrow scope. In a multi-skill environment, over-broad routing can override more appropriate skills, leading to unintended actions, confused task handling, or generation of evaluation artifacts when the user wanted something else.
- Content
Natural-Language Policy Violations
Medium
- Category
- Not specified by scanner
- Confidence
- 90% confidence
- Finding
The module docstring and all visible CLI messages are Chinese-only, which can impose a specific language on users without opt-in. The policy explicitly flags locale/language constraints when the skill does not offer a user choice or justify the restriction.
- Content
Tainted flow: 'cases' from pathlib.Path.read_text (line 23, file read) → pathlib.Path.write_text (file write)
Medium
- Category
- Data Flow
- Confidence
- 65% confidence
- Finding
Data from a source is assigned to a variable that is later passed to a sink, creating a variable-mediated taint flow.
- Content
python return out_p = pathlib.Path(a.out) out_p.parent.mkdir(parents=True, exist_ok=True) out_p.write_text( "\n".join(json.dumps(c, ensure_ascii=False) for c in cases), encoding="utf-8") print(f"已生成 {len(cases)} 条评测样本 → {a.out}")
Natural-Language Policy Violations
Low
- Category
- Not specified by scanner
- Confidence
- 84% confidence
- Finding
SQP-3 适用于所有文件。标题与正文直接以中文编写，并在 L03 指示读者优先阅读另一个文档，但未说明是否支持其他语言或依据用户语言偏好切换，可能构成未获用户选择的语言/locale 约束。
- Content
Natural-Language Policy Violations
Low
- Category
- Not specified by scanner
- Confidence
- 79% confidence
- Finding
The primary trigger and invocation descriptions are framed around Chinese-language user utterances, which can impose a language expectation without explicit opt-in. Although there is bilingual documentation, the activation guidance itself does not clearly state that users may invoke the skill in either language.
- Content
Intent-Code Divergence
Low
- Category
- Not specified by scanner
- Confidence
- 86% confidence
- Finding
The English usage text says the script
bench_build.pyruns the core logic, which implies that file/path directly, while the actual interface declaration points toscripts/bench_build.pyas the entry point. This is a documentation-to-implementation mismatch in the referenced executable location, even though the overall purpose remains aligned.- Content
Intent-Code Divergence
Low
- Category
- Not specified by scanner
- Confidence
- 86% confidence
- Finding
The Chinese usage and workflow sections describe invoking
bench_build.py, but the manifest's actual interface entry point isscripts/bench_build.py. This is an intent/documentation divergence about how the skill is actually executed.- Content
Natural-Language Policy Violations
Low
- Category
- Not specified by scanner
- Confidence
- 88% confidence
- Finding
The natural-language content presents the skill index and trigger instructions entirely in Chinese, including the directive in L3 and activation guidance in L13, with no indication that users may choose another language. This can violate a language-choice policy when no opt-in or justification is provided.
- Content