Back to skill

Security audit

Playwright Test Generator

Security checks for vulnerabilities and agentic risk

Overview

This is a coherent Playwright test generator, but it can browse arbitrary URLs and produce executable test files from unescaped user or page data, so it should be reviewed before installation.

Install only if you are comfortable reviewing generated tests before running them, especially in CI. Use mock or trusted test URLs, avoid pointing it at internal or production systems, do not feed untrusted HTML or test data without review, and update/pin dependencies before use.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (3)

T09 · Insecure Skill Coding Practices

Error
Location
playwright_test_generator.py:192
Finding

Unrestricted URL Analysis Enables Server-Side Request Forgery

Content
View full analysis
str: """Generate test code from a URL.""" from playwright.sync_api import sync_playwright code_blocks = [] with sync_playwright() as p: browser = p.chromium.launch(headless=True) context = browser.new_context() page = context.new_page() try: page.goto(url, wait_until='domcontentloaded', timeout=30000) page.wait_for_timeout(1000) analyzer = PageAnalyzer(page) if pom: code_blocks.append(_generate_pom_code(analyzer, language)) else: code_blocks.append(_generate_test_code(analyzer, language)) finally: page.close() context.close() browser.close() return '\n\n'.join(code_blocks) ``` The JavaScript implementation has the same weakness: ```javascript async analyzeUrl(url, options = {}) { if (!url) throw new Error('URL is required for analyzeUrl'); const { mock = false } = options; if (mock) { return { url, title: 'Mock Page Title', locators: { 'main-heading': 'h1', 'login-form': '#login-form', 'username-input': '[data-testid="username-input"]', 'password-input': '[data-testid="password-input"]', 'submit-button': '[data-testid="submit-button"]' }, elements: [] }; } const { chromium } = await import('playwright'); const browser = await chromium.launch({ headless: true }); const page = await browser.newPage(); await page.goto(url, { waitUntil: 'domcontentloaded' }); const html = await page.content(); const title = await page.title() ...[truncated 2528 chars]
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Error
Location
src/parser.js:314
Finding

Unescaped Inputs Permit Arbitrary Code Injection into Generated Tests

Content
View full analysis
` { ${Object.entries(d).map(([k, v]) => `${k}: '${v}'`).join(', ')} }`).join(',\n')} ]; ``` Locator names and selectors are directly interpolated: ```javascript const locatorsCode = Object.entries(locators).map(([name, sel]) => { const method = name.replace(/[\s_-]+/g, '_').toLowerCase( ...[truncated 3591 chars]
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Warning
Location
src/generator.js:163
Finding

Generated Data-Driven Tests Expose Credential Records in Logs

Content
View full analysis
` { ${Object.entries(d).map(([k, v]) => `${k}: '${v}'`).join(', ')} }`).join(',\n')} ]; for (const data of TEST_DATA) { test(data.username + ' login test', async ({ page }) => { await page.goto('https://example.com'); // Use data.username, data.password etc. console.log(data); }); } `; } ``` ### Technical Analysis The generated test explicitly anticipates records containing fields such as `username` and `password`, then logs each complete record with `console.log(data)`. Data-driven test records commonly contain passwords, authentication tokens, API keys, email addresses, session data, or other sensitive values. Logging the complete object exposes every field regardless of whether it is necessary for test diagnostics. The data is written when the generated test runs, so exposure can occur in local terminal history, CI job output, centralized logging platforms, test reports, or retained build artifacts. ### Attack Path 1. A user supplies a test-data JSON file containing credentials or other secrets. 2. The generator embeds those records in a Jest data-driven test. 3. The generated test is executed locally or in CI. 4. `console.log(data)` writes every field, including passwords or tokens, to standard output. 5. CI or logging infrastructure stores and distributes the output according to its retention and access policies. 6. Any person or integra ...[truncated 611 chars]
Remediation
View remediation
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Memory PoisoningPersistent Context Injection, Context Window Stuffing, Memory Manipulation
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
Findings (37)

Tp4

High
Category
MCP Tool Poisoning
Confidence
93% confidence
Finding

The declared description presents a broad AI-driven Playwright test generation skill, including generating multiple kinds of tests from natural language, HTML, or URLs across several frameworks. The supplied code chunk is much narrower: it only parses provided HTML, extracts selectors, offers selector lookup, and emits a simple JavaScript Page Object class. While this partially aligns with the claimed ability to build POMs from HTML analysis, it does not implement the larger primary purpose of generating full test scripts or data-driven tests, nor does it show AI/natural-language handling or URL-based analysis. This is therefore a material description-to-behavior mismatch.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
89% confidence
Finding

The supplied code is clearly related to Playwright test generation, so it broadly aligns with the domain of the description. However, the declared description significantly overstates the implemented capabilities. This chunk only contains static template generators and utility functions for assembling test code strings, POM scaffolding, and Gherkin text. It does not contain any AI logic, parsing of natural language, HTML inspection, URL fetching/analysis, or data-driven test generation. It also does not show support for a separate 'native Playwright' mode beyond the Python and JavaScript template families explicitly present. Additionally, it contains Gherkin/BDD generation, which is outside the declared purpose. Because the actual implemented behavior is materially narrower and somewhat different from the declared capabilities, this should be flagged as a mismatch.

Content

No source excerpt is available for this finding.

Known Vulnerable Dependency: brace-expansion==2.0.3 — 3 advisory(ies): CVE-2026-13149 (brace-expansion: DoS via exponential-time expansion of consecutive non-expanding); CVE-2026-14257 (brace-expansion: DoS via unbounded expansion length causing an out-of-memory pro); CVE-2026-69152 (brace-expansion: DoS via unbounded intermediate arrays, bypassing the CVE-2026-1)

High
Category
Supply Chain
Confidence
91% confidence
Finding

brace-expansion 2.0.3 has multiple DoS issues involving exponential or unbounded expansion that can consume CPU or memory. Even though this instance is likely transitive, any code path that expands attacker-influenced glob or brace patterns can be crashed or severely slowed, which is especially relevant in developer tooling and automation contexts.

Content

No source excerpt is available for this finding.

Known Vulnerable Dependency: browserslist==4.28.2 — 2 advisory(ies): CVE-2026-73088 (Browserslist: Uncaught crash / prototype write via untrusted browserslist-stats.); CVE-2026-73089 (Browserslist: Unbounded memory growth (no cache eviction) via distinct query res)

High
Category
Supply Chain
Confidence
83% confidence
Finding

browserslist 4.28.2 is flagged for crash/prototype-write and memory growth issues when handling untrusted browserslist-stats or distinct queries. In this project it is a dev-tooling dependency, which lowers runtime exposure, but if CI, code generation, or tooling consumes attacker-controlled config or stats files, this can still cause denial of service or unsafe object mutation.

Content

No source excerpt is available for this finding.

Known Vulnerable Dependency: brace-expansion==1.1.13 — 3 advisory(ies): CVE-2026-13149 (brace-expansion: DoS via exponential-time expansion of consecutive non-expanding); CVE-2026-14257 (brace-expansion: DoS via unbounded expansion length causing an out-of-memory pro); CVE-2026-69152 (brace-expansion: DoS via unbounded intermediate arrays, bypassing the CVE-2026-1)

High
Category
Supply Chain
Confidence
90% confidence
Finding

brace-expansion 1.1.13 carries the same class of CPU and memory exhaustion vulnerabilities as newer affected branches. Because it is nested under other tooling dependencies, exploitation would most likely occur through attacker-controlled glob patterns in test/build automation, resulting in denial of service rather than direct code execution.

Content

No source excerpt is available for this finding.

Known Vulnerable Dependency: js-yaml==3.14.2 — 4 advisory(ies): CVE-2026-84375 (js-yaml: maxTotalMergeKeys does not limit CPU use for empty merge sources); CVE-2026-59869 (js-yaml: YAML merge-key chains can force quadratic CPU consumption); GHSA-5p4m-2wfm-xmqj (JS-YAML: Quadratic CPU consumption in !!omap resolution (3.x and 4.x) — CVE-2026) +1 more

High
Category
Supply Chain
Confidence
89% confidence
Finding

js-yaml 3.14.2 is affected by multiple quadratic or otherwise excessive CPU-consumption issues when parsing crafted YAML structures. Although present as a dev dependency, YAML parsing in developer tooling or CI commonly touches semi-trusted project files, so a malicious config file could hang or significantly degrade automation workflows.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The design explicitly supports taking a user-supplied URL, launching a browser, navigating to it, and extracting DOM content, but it does not document any trust boundaries, URL validation, network safety controls, or warnings about interacting with external/live systems. In a test-generation skill, this can cause the agent to visit attacker-controlled or production URLs, triggering outbound requests, script execution in the browser context, and unintended actions against real services during analysis.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
89% confidence
Finding

The README encourages URL analysis and writing generated output to files but does not warn users that these actions may trigger outbound network requests or filesystem modifications. In an AI-agent or automation context, that omission can lead to unintended access to internal URLs, SSRF-like behavior through agent tooling, or overwriting local files when users invoke commands based on the documentation.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
88% confidence
Finding

The skill advertises capabilities that imply reading files, writing generated code, and accessing URLs, but it does not declare any explicit tool scope or permissions boundaries. In an agent environment, missing scope declarations can allow broader-than-expected tool access or make reviewers unable to verify whether file and network operations are appropriately constrained.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

Overly broad activation language can cause the skill to trigger on unrelated requests and start handling inputs outside its intended QA/test-generation scope. In an agent system, this can lead to inappropriate use of file/network capabilities, accidental data exposure, or generation of unreviewed code in contexts the user did not intend.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
89% confidence
Finding

Ambiguous 'when to use' conditions broaden the skill's activation surface and increase the chance of misrouting prompts to a skill that may read files, analyze HTML, or fetch URLs. Because this skill's context involves potentially sensitive page content and code generation, accidental invocation is more dangerous than for a purely informational skill.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
83% confidence
Finding

In the batch path, the CLI automatically saves every generated output to disk via output.save(filepath) and only reports the save after it happens. Unlike a clearly destructive operation, file creation is expected for this tool, but this command lacks any pre-write disclosure or confirmation in the code path itself while performing repeated file writes.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The generator emits test steps that will navigate to arbitrary user-supplied URLs and then perform live fills and clicks, which can trigger real side effects such as form submissions, account actions, purchases, or workflow changes when the generated code is executed. In the context of a Playwright test-generation skill, this is more dangerous because the tool is explicitly designed to automate browser interactions and may be pointed at production systems without guardrails or user confirmation.

Content

No source excerpt is available for this finding.

Context Window Stuffing

Medium
Category
Memory Poisoning
Confidence
85% confidence
Finding

Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Content

Scanner excerpt · playwright_test_generator.py (reported line 438)May include surrounding context.

python
'open': 'Navigate to page',
        'navigate': 'Navigate to page',
        'go': 'Navigate to page',
        'fill': 'Fill input field',
        'enter': 'Enter data',
        'type': 'Type data',
        'click': 'Click element',

Context Window Stuffing

Medium
Category
Memory Poisoning
Confidence
85% confidence
Finding

Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Content

Scanner excerpt · playwright_test_generator.py (reported line 468)May include surrounding context.

python
'open': 'Navigate to page',
        'navigate': 'Navigate to page',
        'go': 'Navigate to page',
        'fill': 'Fill input field',
        'enter': 'Enter data',
        'type': 'Type data',
        'click': 'Click element',

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

Allowing invocation whenever the user merely mentions Playwright test generation creates ambiguity in tool selection and increases the chance of unintentional activation. Because this skill supports URL and HTML analysis plus code artifact generation, accidental invocation expands the attack surface and may process external or untrusted content unexpectedly.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

Allowing invocation whenever the user merely mentions Playwright test generation creates ambiguity in tool selection and increases the chance of unintentional activation. Because this skill supports URL and HTML analysis plus code artifact generation, accidental invocation expands the attack surface and may process external or untrusted content unexpectedly.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The skill invites users to supply URLs or HTML for analysis but does not warn that this may involve processing untrusted external content and producing generated files or locators from it. In practice, this can surprise users, increase exposure to prompt-injection-style content embedded in HTML, and create downstream risks if generated artifacts are trusted or executed without review.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The docstring for generate_page_object says it generates a complete Page Object Model class, and the surrounding templates imply real POM methods will be rendered. However, the implementation builds method_code from PythonTemplates.generate_step_screenshot.__doc__, which inserts the text Generate a screenshot step. for each method rather than generating the supplied method definitions.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The generate_scenario docstring states it generates a Gherkin scenario, and the class also defines step_when, step_then, and step_and, suggesting support for varied step types. In practice, generate_scenario formats every input step through GHERKIN_STEP, whose template hardcodes Given, so When and Then style steps are contradicted by the implementation.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Low
Category
Not specified by scanner
Confidence
97% confidence
Finding

The section is explicitly labeled as pytest-playwright Python output, but the example methods contain await inside non-async def functions and include JavaScript-style semicolons, which is not valid Python. This is an intent/documentation contradiction in the README example rather than an implementation detail.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
95% confidence
Finding

The info command presents subscription pricing only in yen (¥), which imposes a locale-specific presentation in natural-language output without opt-in or explanation. This can violate language/locale policy expectations when the tool is not clearly region-specific.

Content

No source excerpt is available for this finding.

Known Vulnerable Dependency: @babel/core==7.29.0 — 1 advisory(ies): CVE-2026-49356 (@babel/core: Arbitrary File Read via sourceMappingURL Comment)

Low
Category
Supply Chain
Confidence
77% confidence
Finding

@babel/core 7.29.0 is flagged for an arbitrary file read via sourceMappingURL handling. In this lockfile it appears only as a development/test dependency, so exploitability in the deployed skill is limited, but if untrusted JavaScript is processed during local builds, tests, or CI, it could expose local files accessible to the build runner.

Content

No source excerpt is available for this finding.

Known Vulnerable Dependency: baseline-browser-mapping==2.10.16 — 1 advisory(ies): CVE-2026-45819 (baseline-browser-mapping process termination on invalid input causes denial of s)

Low
Category
Supply Chain
Confidence
65% confidence
Finding

baseline-browser-mapping 2.10.16 is reported as susceptible to process termination on invalid input, causing denial of service. It is a transitive dev dependency and not obviously part of runtime skill execution, so the real-world risk is mostly limited to build or test workflows that accept attacker-controlled input.

Content

No source excerpt is available for this finding.

Unpinned Dependencies

Low
Category
Supply Chain
Confidence
40% confidence
Finding

Dependencies lack version pinning, allowing potential malicious package updates. Consider pinning versions.

Content

Scanner excerpt · package.json (reported line 29)May include surrounding context.

json
"author": "kay",
  "license": "MIT",
  "dependencies": {
    "cheerio": "^1.0.0",
    "ejs": "^3.1.9",
    "playwright": "^1.42.0"
  },