Back to skill

Security audit

browser-testing-toolkit

Security checks for vulnerabilities and agentic risk

Overview

This browser testing skill is mostly purpose-aligned, but it deserves Review because it can run arbitrary local server commands through a shell and can automatically click consent controls without enough user scoping.

Install only if you trust the projects and instructions it will run against. Treat any server command passed to scripts/with_server.py as full local command execution, avoid copying commands from untrusted pages or repositories, and require explicit approval before starting servers or auto-accepting cookie/consent banners.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Error
Location
scripts/with_server.py:69
Finding
Shell Command Injection Through User-Controlled Server Command## Vulnerability Details **File Location**: `scripts/with_server.py`, lines 69–74 **Vulnerability Type**: OS command injection through unsafe shell invocation **Risk Level**: High ### Vulnerable Code ```python # Use shell=True to support commands with cd and && process = subprocess.Popen( server['cmd'], shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE ) ``` ### Technical Analysis The value supplied through the repeatable `--server` command-line argument is passed directly to `subprocess.Popen` with `shell=True`. Consequently, the system shell interprets the entire value, including command separators, redirections, substitutions, pipelines, and other shell syntax. Arbitrary server command execution is part of the utility's declared functionality. Nevertheless, this implementation lacks a security boundary between an intended executable and additional shell operations. It becomes exploitable when an agent or wrapper constructs `--server` from untrusted repository content, browser content, configuration, or task input. The explicit support for `cd ... && ...` does not require exposing unrestricted shell interpretation; a working directory can instead be passed through `cwd`, while the executable and its arguments can be supplied as an argument array. ### Attack Path 1. An attacker places a crafted server command in repository documentation, configuration, test instructions, or other content that may influence an automated agent. 2. The agent uses that value as the `--server` argument to `scripts/with_server.py`. 3. The script assigns the argument to `server['cmd']`. 4. `subprocess.Popen(..., shell=True)` forwards the complete string to the operating-system shell. 5. The shell interprets attacker-supplied operators or substitutions and executes unintended commands alongside or instead of the development server. 6. Those commands run with the same operating-system identity ...[truncated 715 chars]
Remediation
## Remediation Suggestions 1. Remove `shell=True` and execute commands as explicit argument arrays with `shell=False`. 2. Add a dedicated `--cwd` option rather than supporting `cd DIRECTORY && COMMAND` through shell syntax. 3. Define server commands using a structured format containing the executable, arguments, and working directory. For example: ```python process = subprocess.Popen( server_argv, shell=False, cwd=server_cwd, stdout=subprocess.PIPE, stderr=subprocess.PIPE ) ``` 4. Never construct executable arguments from browser DOM content, remote responses, repository instructions, or other untrusted text without validation and explicit user approval. 5. If shell syntax must be retained for compatibility, make it an explicit opt-in mode, clearly warn that it grants arbitrary command execution, and require confirmation before running commands derived from external or project-controlled content. 6. Where practical, validate executables against an allowlist appropriate to the testing environment and run the utility in a restricted account or sandbox with minimal filesystem, credential, and network access.
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Tool MisuseTool Parameter Abuse, Chaining Abuse, Unsafe Defaults
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
Findings (12)

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding
The skill is presented as a browser testing toolkit, but its documented behavior includes general-purpose subprocess execution and lifecycle management via with_server.py, including starting arbitrary commands and managing process cleanup. That mismatch can cause reviewers or policy layers to underestimate the skill's real execution power, enabling broader local command execution than the description suggests.

Tool Parameter Abuse

High
Category
Tool Misuse
Content
print(f"Starting server {i+1}/{len(servers)}: {server['cmd']}")

            # Use shell=True to support commands with cd and &&
            process = subprocess.Popen(
                server['cmd'],
                shell=True,
                stdout=subprocess.PIPE,
Confidence
99% confidence
Finding
The finding is duplicated but valid: the code exposes a high-risk execution sink by combining an externally controllable parameter with shell=True. An attacker who can modify invocation arguments, task definitions, or repository scripts could run arbitrary commands, exfiltrate secrets, or alter the test environment.

Tool Parameter Abuse

High
Category
Tool Misuse
Content
print(f"Starting server {i+1}/{len(servers)}: {server['cmd']}")

            # Use shell=True to support commands with cd and &&
            process = subprocess.Popen(
                server['cmd'],
                shell=True,
                stdout=subprocess.PIPE,
Confidence
99% confidence
Finding
The finding is duplicated but valid: the code exposes a high-risk execution sink by combining an externally controllable parameter with shell=True. An attacker who can modify invocation arguments, task definitions, or repository scripts could run arbitrary commands, exfiltrate secrets, or alter the test environment.

Lp3

Medium
Category
MCP Least Privilege
Confidence
86% confidence
Finding
The skill exposes shell-capable behavior such as running Playwright CLI, Python scripts, and server orchestration commands, but it declares no explicit tool scope or allowed-tools boundary. In an agent environment, this increases the chance of over-broad execution and makes it harder to constrain what the skill may invoke, especially since the document includes arbitrary command examples like npm run dev and python server.py.

Vague Triggers

Medium
Confidence
90% confidence
Finding
The statement that the skill will automatically choose the optimal branch based on task type lacks precise boundaries, priority rules, and safety gates. Because the branches include browser automation, DevTools inspection, and Python/server orchestration, vague routing logic can escalate a simple request into more powerful actions than necessary.

Vague Triggers

Medium
Confidence
96% confidence
Finding
The trigger keywords are very broad terms like click, fill, simple, debug, and test, which can overlap with many ordinary conversations and unrelated requests. In an agentic system this can cause unintended skill activation, leading to unnecessary browser actions or shell-backed test execution in contexts where the user did not intend to invoke this capability.

External Transmission

Medium
Category
Data Exfiltration
Content
```bash
# 路由拦截
playwright-cli route "**/*.jpg" --status=404
playwright-cli route "https://api.example.com/**" --body='{"mock": true}'
```

---
Confidence
50% confidence
Finding
Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Missing User Warnings

Medium
Confidence
93% confidence
Finding
The code automatically clicks consent or acceptance controls on cookie banners and similar overlays without explicit user approval or a restrictive policy. In a browser automation toolkit, this can silently perform legally or privacy-significant actions on behalf of the user, potentially accepting tracking, consent terms, or site preferences the operator did not intend.

subprocess module call

Medium
Category
Dangerous Code Execution
Content
print(f"Starting server {i+1}/{len(servers)}: {server['cmd']}")

            # Use shell=True to support commands with cd and &&
            process = subprocess.Popen(
                server['cmd'],
                shell=True,
                stdout=subprocess.PIPE,
Confidence
98% confidence
Finding
The script launches server commands with subprocess.Popen(..., shell=True) using values taken directly from --server arguments. If an untrusted actor can influence that parameter, shell metacharacters, command chaining, variable expansion, or redirection can execute arbitrary OS commands under the user's privileges.

subprocess module call

Medium
Category
Dangerous Code Execution
Content
# Run the command
        print(f"Running: {' '.join(args.command)}\n")
        result = subprocess.run(args.command)
        sys.exit(result.returncode)

    finally:
Confidence
70% confidence
Finding
subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Natural-Language Policy Violations

Low
Confidence
85% confidence
Finding
描述文本直接以中文给出核心说明,全文也主要使用中文,但未说明是否会根据用户偏好切换语言或先征求语言选择。根据语言/地区策略,若技能默认强制单一语言而无用户选择,可能构成自然语言政策问题。

Natural-Language Policy Violations

Low
Confidence
87% confidence
Finding
The primary natural-language documentation and usage descriptions in this file are presented in Chinese, with no indication that language is configurable or intentionally limited to a Chinese-only audience. Under the policy, forcing a specific language without user choice can be a locale/language policy violation.

Static analysis

No suspicious patterns detected.