Webapp Testing

Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing...

Security Scan

The skill is internally consistent with a Playwright-based local webapp testing toolkit, but it includes a helper that runs shell commands (shell=True) and the instructions encourage treating scripts as black boxes — review before running.

Purpose & Capability

Name/description match the provided artifacts: Playwright examples and a server helper. No unrelated environment variables, binaries, or install steps are requested. The helper script's ability to start dev servers is appropriate for a local webapp testing toolkit.

Instruction Scope

SKILL.md stays within testing/useful automation patterns (take screenshots, inspect DOM, wait for networkidle). However it explicitly recommends treating bundled scripts as black boxes and not reading source unless necessary — that guidance reduces opportunity for user audit. The runtime instructions and examples show the helper starting arbitrary shell commands and writing files under /mnt/user-data and /tmp; these behaviors are expected for the stated purpose but worth reviewing before execution.

Install Mechanism

No install spec — instruction-only with included example scripts. This minimizes install-time risks (nothing downloaded or extracted). The code is bundled in the skill, so nothing external is fetched at install-time.

Credentials

The skill requests no environment variables or credentials. Example scripts write outputs to /mnt/user-data and /tmp and use localhost ports; these are reasonable for local testing and consistent with the skill's purpose.

Persistence & Privilege

always is false and the skill does not request persistent privileges or modify other skills. Model invocation is enabled (default) which is normal; note that autonomous invocation combined with shell-capable helpers increases blast radius and should be considered by administrators.

Scan Findings in Context

[subprocess-shell-usage] expected: scripts/with_server.py uses subprocess.Popen with shell=True to support complex commands like 'cd ... && npm run dev'. This is expected for a helper that accepts arbitrary shell commands, but shell=True makes it dangerous to run with untrusted inputs or without reviewing the command being executed.

[arbitrary-process-execution] expected: with_server.py launches server processes and then runs a user-specified command; this matches the stated purpose (starting servers then running automation). Still, it executes arbitrary commands given to it, so commands should come from trusted sources.

[writes-to-shared-paths] expected: Examples write outputs to /mnt/user-data/outputs and /tmp which is expected for test artifacts. Confirm these paths are acceptable in your environment.

Assessment

This skill appears to be what it says: Playwright examples plus a helper to start dev servers. Before installing or running it: 1) Inspect scripts/with_server.py — it spawns user-supplied shell commands (subprocess with shell=True); do not supply untrusted commands. 2) Run the helper with --help and test in an isolated environment (container or VM) so a misbehaving dev server or script can't affect your host. 3) If you plan to allow autonomous agent use, restrict or review any commands the agent can pass to the helper (autonomous invocation + shell-capable helpers increases risk). 4) Check where outputs are written (/mnt/user-data, /tmp) and adjust paths or permissions if needed. 5) If you want stronger safety, run the scripts manually the first time and audit the code (contrary to the SKILL.md suggestion to avoid reading source).

Web Application Testing

To test local web applications, write native Python Playwright scripts.

Helper Scripts Available:

• scripts/with_server.py - Manages server lifecycle (supports multiple servers)

Always run scripts with --help first to see usage. DO NOT read the source until you try running the script first and find that a customized solution is abslutely necessary. These scripts can be very large and thus pollute your context window. They exist to be called directly as black-box scripts rather than ingested into your context window.

Decision Tree: Choosing Your Approach

User task → Is it static HTML?
    ├─ Yes → Read HTML file directly to identify selectors
    │         ├─ Success → Write Playwright script using selectors
    │         └─ Fails/Incomplete → Treat as dynamic (below)
    │
    └─ No (dynamic webapp) → Is the server already running?
        ├─ No → Run: python scripts/with_server.py --help
        │        Then use the helper + write simplified Playwright script
        │
        └─ Yes → Reconnaissance-then-action:
            1. Navigate and wait for networkidle
            2. Take screenshot or inspect DOM
            3. Identify selectors from rendered state
            4. Execute actions with discovered selectors

Example: Using with_server.py

To start a server, run --help first, then use the helper:

Single server:

python scripts/with_server.py --server "npm run dev" --port 5173 -- python your_automation.py

Multiple servers (e.g., backend + frontend):

python scripts/with_server.py \
  --server "cd backend && python server.py" --port 3000 \
  --server "cd frontend && npm run dev" --port 5173 \
  -- python your_automation.py

To create an automation script, include only Playwright logic (servers are managed automatically):

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True) # Always launch chromium in headless mode
    page = browser.new_page()
    page.goto('http://localhost:5173') # Server already running and ready
    page.wait_for_load_state('networkidle') # CRITICAL: Wait for JS to execute
    # ... your automation logic
    browser.close()

Reconnaissance-Then-Action Pattern

1. Inspect rendered DOM:

   page.screenshot(path='/tmp/inspect.png', full_page=True)
   content = page.content()
   page.locator('button').all()
   

2. Identify selectors from inspection results

3. Execute actions using discovered selectors

Common Pitfall

❌ Don't inspect the DOM before waiting for networkidle on dynamic apps ✅ Do wait for page.wait_for_load_state('networkidle') before inspection

Best Practices

• Use bundled scripts as black boxes - To accomplish a task, consider whether one of the scripts available in scripts/ can help. These scripts handle common, complex workflows reliably without cluttering the context window. Use --help to see usage, then invoke directly.
• Use sync_playwright() for synchronous scripts
• Always close the browser when done
• Use descriptive selectors: text=, role=, CSS selectors, or IDs
• Add appropriate waits: page.wait_for_selector() or page.wait_for_timeout()

Reference Files

• examples/ - Examples showing common patterns:
  • element_discovery.py - Discovering buttons, links, and inputs on a page
  • static_html_automation.py - Using file:// URLs for local HTML
  • console_logging.py - Capturing console logs during automation