Back to skill

Security audit

Agent Eval Suite

Security checks for vulnerabilities and agentic risk

Overview

This is a coherent local agent-evaluation toolkit, with no hidden persistence or network behavior, but users should treat HTML report inputs and scenario file paths as trusted.

Install only if you are comfortable using a small local Node evaluation library. Avoid generating HTML reports from untrusted titles, metric names, or result data unless the output is escaped or sanitized first, and do not pass untrusted or user-controlled absolute paths to Simulator.loadScenario. The package does not show hidden installation behavior, network exfiltration, credential use, or persistent agent modification.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Warning
Location
src/report-generator.js:576
Finding

Unescaped Report Data Allows HTML and Script Injection

Content
View full analysis
${report.title} body { font-family: Arial, sans-serif; max-width: 1200px; margin: 0 auto; padding: 20px; } h1 { color: #333; } h2 { color: #666; border-bottom: 1px solid #ddd; padding-bottom: 10px; } table { border-collapse: collapse; width: 100%; margin: 20px 0; } th, td { border: 1px solid #ddd; padding: 8px; text-align: left; } th { background-color: #f5f5f5; } .passed { color: green; } .failed { color: red; } .warning { color: orange; } ${md.replace(/\n\n/g, '

').replace(/\n/g, '
').replace(/^# (.*)$/m, '

$1

').replace(/^## (.*)$/m, '

$1

').replace(/^### (.*)$/m, '

$1

')} `; } ``` ### Technical Analysis The HTML generator directly interpolates `report.title` into the `` element and inserts Markdown-derived content into the document body without HTML escaping or sanitization. The Markdown conversion consists only of regular-expression replacements and does not neutralize raw HTML, event-handler attributes, or other active browser content. The public report-generation methods accept caller-provided titles through their `options` parameters. Report-derived fields, including metric and category names, can also be incorporated into Markdown before reaching `toHTML()`. Consequently, an attacker who controls report options or evaluation data can place arbitrary markup into the generated HTML. For example, a malicious title could contain: ```html ``` Likewise, attacker-controlled me ...[truncated 1814 chars]
Remediation
View remediation
`, `"`, and `'` before placing untrusted text into HTML. - Encode the title separately for its `` context. - Encode all metric names, category names, recommendation text, and other report-derived values before rendering them. 2. **Use a maintained Markdown renderer** - Replace the regular-expression-based Markdown conversion with a maintained Markdown library. - Configure the renderer to reject, escape, or disable raw HTML. 3. **Sanitize the final document** - If limited HTML must be supported, process generated content with a maintained HTML sanitizer. - Use a restrictive allowlist of required elements and attributes. - Reject event-handler attributes, executable URLs, scripts, embedded frames, and other active content. 4. **Separate templates from data** - Use a template engine that enables auto-escaping by default. - Avoid marking report-derived values as trusted or raw HTML. 5. **Add defense-in-depth controls** - When reports are served over HTTP, apply a restrictive Content Security Policy that disallows inline scripts and untrusted resource origins. - Prefer serving generated reports from an isolated origin without access to authenticated application data. 6. **Add security regression tests** - Test titles, metric names, category names, and recommendation fields with payloads such as: ```html <img src=x onerror=alert(1)> ``` - Assert that the output contains encoded text rather than executable markup. - Test injection into both the document title and body. ]]>
Vulnerability Patterns
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
Findings (9)

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The main descriptive content is presented in Chinese while the file title and code examples are otherwise accessible to a broader audience, and there is no indication that the skill is intentionally region-specific. This creates a natural-language policy concern because the skill effectively imposes a language choice without user opt-in or justification.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The natural-language description is written entirely in Chinese, which can impose a specific language on users without any opt-in or alternative locale indicated. This matches the policy category for language or locale constraints that are not documented as optional or region-specific.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

This code contains natural-language content in both Chinese comments/documentation and hard-coded English recommendation strings, but provides no mechanism for selecting or opting into a preferred language. That can violate language/locale policy because users may receive output in a forced language rather than a chosen one.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

This JavaScript file contains natural-language documentation comments in Chinese, such as the file header and method descriptions, which effectively impose a specific language on maintainers or users of the skill without any visible opt-in or justification. The policy explicitly allows locale constraints only when they are documented and justified, which is not present here.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The file-level natural-language description is written only in Chinese, while later user-facing recommendation strings are in English. This creates an implicit language constraint without any documented user choice or locale justification, which matches the policy category for language/locale violations.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
98% confidence
Finding

The HTML renderer interpolates report.title and converted markdown-derived content directly into an HTML template without escaping or sanitization. Because report fields are built from caller-controlled results/options data, an attacker could inject HTML or script content that executes when the generated report is viewed, resulting in stored or reflected XSS depending on how reports are distributed.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The module advertises a sandboxed simulation environment, but loadScenario() accepts an absolute path and reads it directly with fs.readFileSync(). If untrusted input can reach scenarioName, a caller can cause arbitrary local file reads outside the intended scenarios directory, violating the sandbox claim and potentially exposing sensitive files or application secrets.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The top-level natural-language description is written only in Chinese ('模拟环境测试', '提供沙箱环境、场景模拟、边界条件测试和故障注入') while the file otherwise contains English identifiers and messages. This creates a locale/language constraint without any opt-in, alternative language option, or documented region-specific justification.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
91% confidence
Finding

The file-level docstrings and many function comments are written in Chinese, while generated report titles and output labels are primarily English, indicating mixed-language behavior without any documented language choice or locale setting. This creates a natural-language policy concern because the skill implicitly assumes a specific language for maintainers or operators without offering opt-in or documenting a justified locale restriction.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.