Website Usability Testing using Nova Act

• Outcome: Review
• Latest audit: May 28, 2026 · 12:35 PM UTC
• Version: 1.0.3

Overview

This skill mostly does what it says, but it can automate live checkout/booking/posting workflows and may send page-derived data to Anthropic without clear opt-in.

Install only if you are comfortable with live browser automation. Prefer staging or test sites, disposable accounts, and synthetic data. Avoid authenticated, private, or production checkout/booking/posting flows unless you accept possible cart/session/form side effects and local capture of screenshots/page content. Do not use default auto-persona mode with ANTHROPIC_API_KEY set unless sending page-derived metadata to Anthropic is acceptable.

SkillSpector

By NVIDIA

Vulnerability Patterns

• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Excessive Agency: Unrestricted Tool Access, Autonomous Decision Making, Scope Creep
• Trigger Abuse: Overly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
• MCP Tool Poisoning: Hidden Instructions, Unicode Deception, Parameter Description Injection
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands

Findings (9)

Intent-Code Divergence

Medium

• Confidence: 78% confidence
• Finding: The document says there are no extra API calls and advises against the script making its own Claude call, yet later states Claude AI may infer personas automatically. That contradiction obscures whether user data, URLs, or page-derived context may be sent to another external model provider, which is a real security and privacy concern.

Description-Behavior Mismatch

Medium

• Confidence: 97% confidence
• Finding: The workflow generator issues action prompts that actively fill forms, add items to carts, navigate to checkout, and otherwise progress through real-world transactions before the final safety stop. In a usability-testing skill, this is dangerous because material-impact actions can still trigger side effects such as cart pollution, reservation holds, account creation attempts, emails/messages, or pre-submission state changes even if the very last confirmation click is avoided.

Context-Inappropriate Capability

Medium

• Confidence: 92% confidence
• Finding: The fallback persona-generation path imports the Anthropic client, reads an API key from the environment, and sends page-derived data to a third-party model. That is a real capability expansion beyond the stated Nova Act usability-testing purpose, and it creates unnecessary data-flow and dependency risk even if the intent appears to be convenience rather than abuse.

Vague Triggers

Medium

• Confidence: 88% confidence
• Finding: The README advertises very broad natural-language trigger phrases like 'test website usability' and 'analyze website UX', which can overlap with ordinary user conversation and cause the skill to activate unexpectedly. For a skill that performs browser automation, collects screenshots/full page content, and writes logs/reports, unintended invocation can expose sensitive site content or launch actions against a website without sufficiently explicit user intent.

Missing User Warnings

Medium

• Confidence: 98% confidence
• Finding: The purchase workflow explicitly instructs the agent to add products to cart, open the cart, proceed to checkout, and fill shipping information before presenting any stop. Those are material-impact actions that can create business-side effects, user confusion, inventory/cart contamination, fraud signals, or unintended order progress, especially in production sites where a later payment page is not the first meaningful commit point.

Missing User Warnings

Medium

• Confidence: 88% confidence
• Finding: The function always writes a full HTML report to the current working directory containing detailed test observations and embeds clickable local file:// trace links. In a usability-testing skill, those traces and notes can include sensitive workflow data, internal paths, screenshots, or credentials entered during testing, and persisting them to disk without explicit consent, redaction, or disclosure increases the risk of unintended local data exposure.

Missing User Warnings

Medium

• Confidence: 90% confidence
• Finding: The code reads a long-lived API key from a local config file and injects it into the process environment, which broadens exposure to subprocesses, crash dumps, debug tooling, and other code running in-process. In a skill that orchestrates browser automation and may invoke third-party libraries, this increases the chance of accidental credential disclosure even if there is no explicit exfiltration logic here.

Missing User Warnings

Medium

• Confidence: 88% confidence
• Finding: The session wrapper automatically creates a logs directory and enables Nova Act trace logging, which can persist sensitive browsing artifacts, prompts, page contents, and workflow data to disk without explicit user consent. Because this skill performs usability testing on real user workflows like checkout and booking, the traces may contain credentials, PII, or transactional details, making disk persistence materially risky.

Missing User Warnings

Medium

• Confidence: 95% confidence
• Finding: The code sends analyzed website title, purpose, and navigation content to Anthropic without an explicit user-facing warning or consent step. In a usability-testing skill, visited pages may include sensitive internal or pre-production content, so silent third-party transmission creates a genuine privacy and confidentiality risk.

VirusTotal

66/66 vendors flagged this skill as clean.

Static analysis

No suspicious patterns detected.