Back to skill

Security audit

Response Speed Test

Security checks for vulnerabilities and agentic risk

Overview

The skill mostly does what it claims, but its report-saving APIs can write outside the intended output folder and overwrite user-writable files if a caller supplies an unsafe filename.

Install only if you are comfortable with a beta performance-testing tool and keep report output under your direct control. Do not pass filenames from prompts, users, or external data into the JSON or Markdown reporter APIs until the package validates paths and prevents overwrites outside the selected report directory.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Warning
Location
reporters/json_export.py:75
Finding
Path Traversal and Arbitrary File Overwrite in Report Writers## Vulnerability Details **File Locations**: - `reporters/json_export.py:75-90` - `reporters/json_export.py:92-110` - `reporters/markdown_report.py:122-137` - `reporters/markdown_report.py:139-157` **Vulnerability Type**: Unrestricted filename handling leading to path traversal and arbitrary file overwrite **Risk Level**: Medium ### Vulnerable Code ```python # reporters/json_export.py:75-90 def save_meter(self, meter: ResponseSpeedMeter, filename: str = None) -> Path: """保存測量結果到文件""" if not self.output_dir: raise ValueError("未設置輸出目錄") self.output_dir.mkdir(parents=True, exist_ok=True) filename = filename or f"response_test_{meter.test_id}.json" filepath = self.output_dir / filename content = self.export_meter(meter) with open(filepath, 'w', encoding='utf-8') as f: f.write(content) return filepath ``` ```python # reporters/json_export.py:92-110 def save_benchmark(self, benchmark: ResponseSpeedBenchmark, filename: str = None) -> Path: """保存基準測試結果到文件""" if not self.output_dir: raise ValueError("未設置輸出目錄") self.output_dir.mkdir(parents=True, exist_ok=True) timestamp = datetime.now().strftime("%Y%m%d_%H%M%S") filename = filename or f"benchmark_{timestamp}.json" filepath = self.output_dir / filename content = self.export_benchmark(benchmark) with open(filepath, 'w', encoding='utf-8') as f: f.write(content) return filepath ``` ```python # reporters/markdown_report.py:122-137 def save_meter(self, meter: ResponseSpeedMeter, filename: str = None) -> Path: """保存測量結果到文件""" if not self.output_dir: raise ValueError("未設置輸出目錄") self.output_dir.mkdir(parents=True, exist_ok=True) filename = filename or f"response_test_{meter.test_id}.md" filepath = self.output_dir / filename content = self.export_detailed_report(meter) with open(fil ...[truncated 3496 chars]
Remediation
## Remediation Suggestions 1. Resolve both the output directory and destination before writing, then enforce destination containment: ```python base = self.output_dir.resolve() base.mkdir(parents=True, exist_ok=True) if Path(filename).is_absolute(): raise ValueError("Absolute report paths are not permitted") target = (base / filename).resolve() try: target.relative_to(base) except ValueError: raise ValueError("Report path escapes the output directory") ``` 2. Restrict caller-provided values to a basename when subdirectories are unnecessary: ```python name = Path(filename) if name.name != filename or ".." in name.parts: raise ValueError("Invalid report filename") ``` 3. Enforce the expected extension, such as `.json` for `JSONReporter` and `.md` for `MarkdownReporter`. 4. Avoid silent replacement of existing files. Use exclusive creation mode unless overwriting is explicitly requested: ```python with open(target, "x", encoding="utf-8") as report_file: report_file.write(content) ``` 5. If intentional overwriting is required, expose a separate `overwrite=False` option and reject existing destinations by default. 6. Where untrusted local users can modify the output directory, mitigate symlink attacks by using a trusted directory with restrictive permissions and platform-appropriate no-follow file-opening controls. 7. Add regression tests covering absolute paths, `../` traversal, nested traversal, unexpected extensions, existing destinations, and symlink destinations.
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
Findings (21)

Natural-Language Policy Violations

Medium
Confidence
92% confidence
Finding
The skill description is written as a mandatory Chinese-language heading and the rest of the README continues in Chinese, with no indication that users may choose another language or that the skill is intentionally limited to a Chinese-speaking region. This can violate language/locale policy requirements that prohibit forcing a specific language without user opt-in.

Natural-Language Policy Violations

Medium
Confidence
95% confidence
Finding
The skill description is written entirely in Traditional Chinese while the title and command examples are in English, with no indication that users can choose their preferred language. This can violate language/locale policy expectations when a skill imposes a language without documenting opt-in or justification.

Natural-Language Policy Violations

Medium
Confidence
96% confidence
Finding
This code presents its docstrings, help text, and runtime console output in Chinese, which imposes a specific language/locale on all users. The file does not provide any opt-in, locale selection, or documented justification for this constraint.

Natural-Language Policy Violations

Medium
Confidence
91% confidence
Finding
The file contains user-facing natural-language documentation in Chinese at L003 while the primary title is in English, but it does not indicate that language selection is optional or region-specific. Per the policy, forcing a specific language or locale without user opt-in is a natural-language policy concern.

Natural-Language Policy Violations

Medium
Confidence
93% confidence
Finding
The file’s docstrings and user-facing descriptive text are written in Traditional Chinese, with no indication that language choice is optional or configurable. Under the stated policy, forcing a specific language without user opt-in is a natural-language policy violation.

Natural-Language Policy Violations

Medium
Confidence
91% confidence
Finding
The module docstring and in-code documentation are written in Chinese, which imposes a specific language for users or maintainers without indicating any language choice or opt-in. The policy explicitly calls for flagging language or locale constraints unless the skill offers a choice or clearly justifies the locale specificity.

External Transmission

Medium
Category
Data Exfiltration
Content
def __init__(self):
        super().__init__(name="llm", stage="T4-T7")
        self.api_url = os.environ.get("LLM_API_URL", "https://api.openai.com/v1")
        self.model = os.environ.get("LLM_MODEL", "gpt-4")
    
    def probe(self, prompt: str = None, **kwargs) -> ProbeResult:
Confidence
60% confidence
Finding
Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Natural-Language Policy Violations

Medium
Confidence
91% confidence
Finding
The module docstrings and user-facing descriptive text are written in Chinese, with no indication that language choice is configurable or optional. This can violate a language/locale policy when skills are expected to respect user language preferences or provide opt-in for non-default locales.

Natural-Language Policy Violations

Medium
Confidence
86% confidence
Finding
Multiple docstrings and comments describe the skill exclusively in Chinese, including the module description and argument documentation. This creates a language/locale policy issue because the file does not provide an opt-in, alternative language, or justification for restricting instructions to a specific language.

Natural-Language Policy Violations

Low
Confidence
90% confidence
Finding
The natural-language description on L03 is written only in Traditional Chinese and provides no indication that language selection is optional or context-specific. This can conflict with the stated policy against forcing a specific language or locale without user opt-in.

Natural-Language Policy Violations

Low
Confidence
89% confidence
Finding
The module description is written only in Traditional Chinese, which can amount to a language/locale constraint in natural-language content without any indication of user opt-in or a documented region-specific reason. The policy for this audit asks to flag language-policy issues when a specific language is effectively imposed without choice.

Natural-Language Policy Violations

Low
Confidence
85% confidence
Finding
This code file contains natural-language strings and documentation exclusively in Chinese, including the primary class and method descriptions. Under the stated policy, forcing a specific language without offering user choice or documenting a justified locale constraint can be a language/locale policy violation.

Unverifiable Dependency: setuptools has 10 known advisory(ies) (CVE-2013-1633 (Setuptools vulnerable to Man-in-the-middle attacks); CVE-2025-47273 (setuptools has a path traversal vulnerability in PackageIndex.download that lead); CVE-2024-6345 (setuptools vulnerable to Command Injection via package URL) +7 more), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low
Category
Supply Chain
Confidence
40% confidence
Finding
Dependency has known vulnerabilities (CVEs). Using packages with unpatched security flaws exposes the environment to known exploits.

Unverifiable Dependency: wheel has 4 known advisory(ies) (CVE-2026-24049 (Wheel Affected by Arbitrary File Permission Modification via Path Traversal in w); CVE-2022-40898 (pypa/wheel vulnerable to Regular Expression denial of service (ReDoS)); CVE-2022-40898 (An issue discovered in Python Packaging Authority (PyPA) Wheel 0.37.1 and earlie) +1 more), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low
Category
Supply Chain
Confidence
40% confidence
Finding
Dependency has known vulnerabilities (CVEs). Using packages with unpatched security flaws exposes the environment to known exploits.

Unverifiable Dependency: pytest has 2 known advisory(ies) (CVE-2025-71176 (pytest has vulnerable tmpdir handling); CVE-2025-71176 (pytest has vulnerable tmpdir handling)), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low
Category
Supply Chain
Confidence
40% confidence
Finding
Dependency has known vulnerabilities (CVEs). Using packages with unpatched security flaws exposes the environment to known exploits.

Unverifiable Dependency: black has 5 known advisory(ies) (CVE-2026-32274 (Black: Arbitrary file writes from unsanitized user input in cache file name); CVE-2024-21503 (Black vulnerable to Regular Expression Denial of Service (ReDoS)); CVE-2024-21503 (Versions of the package black before 24.3.0 are vulnerable to Regular Expression) +2 more), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low
Category
Supply Chain
Confidence
40% confidence
Finding
Dependency has known vulnerabilities (CVEs). Using packages with unpatched security flaws exposes the environment to known exploits.

Natural-Language Policy Violations

Low
Confidence
84% confidence
Finding
The natural-language text in the module header includes Chinese phrases ('報告生成模組', '支持多種輸出格式') with no indication that this is optional or required for a region-specific context. Because the policy applies to all file types and prohibits forcing a specific language without user opt-in, this mixed-language presentation may violate language/locale expectations for users who are not given a choice.

Natural-Language Policy Violations

Low
Confidence
87% confidence
Finding
The natural-language strings in the module docstring and class docstring are written exclusively in Traditional Chinese, with no indication that the user can select another language or that the tool is region-specific. This can conflict with a language/locale policy that requires user choice or documented justification for a fixed locale.

Natural-Language Policy Violations

Low
Confidence
88% confidence
Finding
This Python file contains natural-language docstrings and output text entirely in Traditional Chinese, including the class description and exported report headings. Under the stated policy, forcing a specific language without user opt-in can be a locale-policy violation when no choice or justification is provided.

Natural-Language Policy Violations

Low
Confidence
89% confidence
Finding
The file includes natural-language text in Chinese (for example, the module title) and continues this pattern throughout docstrings and assertions, indicating the skill/test suite expects a specific language context. Under the stated policy, forcing a specific language without user opt-in or clear justification is a natural-language policy concern.

Natural-Language Policy Violations

Low
Confidence
94% confidence
Finding
The test requires the generated comparison report to contain the Chinese phrase "測試次數", which constrains acceptable output to a specific language. This is a natural-language locale policy issue unless the locale restriction is explicitly justified or configurable.

Static analysis

No suspicious patterns detected.