Shopify AI Toolkit

• Outcome: Review
• Latest audit: May 29, 2026 at 9:33 PM UTC
• Version:
  1.1.0

Overview

This Shopify development skill is coherent, but should be reviewed because its required helper scripts send user queries and raw code to Shopify telemetry endpoints by default.

Install only if you are comfortable with Shopify-operated endpoints receiving search terms, validation results, and possibly raw code or file context. Set OPT_OUT_INSTRUMENTATION=true before use in private or regulated projects, avoid passing secrets/customer data into validation, and note that some GraphQL validators appear to reference missing schema asset files in this package.

SkillSpector

By NVIDIA

Vulnerability Patterns

• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Excessive Agency: Unrestricted Tool Access, Autonomous Decision Making, Scope Creep
• MCP Tool Poisoning: Hidden Instructions, Unicode Deception, Parameter Description Injection
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access

Findings (63)

Context-Inappropriate Capability

Medium

• Confidence: 99% confidence
• Finding: The script defines telemetry that POSTs validation results and context to a remote Shopify endpoint, and the caller later passes the full user-supplied `code` into that telemetry context. This creates unsolicited outbound transmission of potentially sensitive source code and metadata from a local validation tool, which is dangerous because users may validate proprietary code, secrets, tokens, or customer data without realizing it will be sent off-host.

Context-Inappropriate Capability

Medium

• Confidence: 98% confidence
• Finding: The script's instrumentation function performs an outbound POST to a Shopify endpoint and is later used to transmit validation context. Because this validator accepts user-supplied code and file metadata, the telemetry path can exfiltrate sensitive source content and environment details off-host without any strong consent mechanism beyond an opt-out environment variable.

Context-Inappropriate Capability

Low

• Confidence: 92% confidence
• Finding: In full-theme mode, the script reports the local theme path and selected file list to a remote service. Even without raw source contents, local filesystem paths and filenames can reveal usernames, project names, internal structure, or other sensitive operational details, creating an unnecessary data exposure channel.

Context-Inappropriate Capability

Medium

• Confidence: 98% confidence
• Finding: The script transmits validation results and the full GraphQL code (`code`) to an external endpoint via `reportValidation`, which is unrelated to the core local validation function. Because the validated operation may contain proprietary queries, schema usage, store identifiers, or business logic, this creates an unexpected data exfiltration path from a developer tool.

Context-Inappropriate Capability

Medium

• Confidence: 95% confidence
• Finding: The script performs remote telemetry from a local validation tool by POSTing validation context and results to a Shopify endpoint. Because the reported payload includes user-provided GraphQL code and metadata, this creates an unexpected data exfiltration path for potentially sensitive queries, schemas, or merchant-specific details.

Context-Inappropriate Capability

Medium

• Confidence: 98% confidence
• Finding: The script defines a telemetry function that sends validation activity to an external endpoint at shopify.dev whenever instrumentation is not explicitly disabled. In a security-sensitive skill, exfiltrating user-supplied validation data and client metadata off-host is dangerous because submitted GraphQL may contain proprietary schema usage, business logic, store identifiers, or other sensitive material.

Context-Inappropriate Capability

High

• Confidence: 100% confidence
• Finding: The outbound telemetry payload includes the raw GraphQL code via the `code` field, along with model/client metadata and artifact identifiers. Raw query text can expose confidential storefront operations, tokens embedded in queries, internal field usage, or tenant-specific business data, making this materially more severe than generic usage analytics.

Context-Inappropriate Capability

Medium

• Confidence: 95% confidence
• Finding: The validation script performs outbound telemetry in `reportValidation`, which is unrelated to the core local validation function and sends execution metadata off-host. In this context it also includes user-supplied GraphQL code in the reported payload, creating a confidentiality risk for queries, schemas, business logic, or secrets accidentally embedded in the operation.

Context-Inappropriate Capability

Medium

• Confidence: 96% confidence
• Finding: The validation script performs network telemetry to a remote endpoint even though its apparent role is local GraphQL validation. It sends the validated GraphQL code and client metadata off-box, which can leak sensitive queries, schema usage, store identifiers, or proprietary business logic entered into the validator.

Intent-Code Divergence

Low

• Confidence: 90% confidence
• Finding: Telemetry is enabled by default and only disabled via an opt-out environment variable, which is inconsistent with the expected behavior of a validator handling potentially sensitive input. This increases the likelihood of accidental data exfiltration because users may not know they must set a flag to prevent reporting.

Context-Inappropriate Capability

Medium

• Confidence: 95% confidence
• Finding: The script's instrumentation helper sends the full GraphQL operation (`code`) plus client metadata to `https://shopify.dev/mcp/usage` by default. Because GraphQL operations can contain sensitive business logic, identifiers, and potentially embedded literals, exfiltrating the full query to a remote service exceeds the apparent local validation purpose and creates an unnecessary data disclosure channel.

Context-Inappropriate Capability

Medium

• Confidence: 98% confidence
• Finding: The script sends user-supplied GraphQL code, API name, model, client name/version, artifact ID, and revision to a remote endpoint via `fetch` in `reportValidation()`. That creates an exfiltration path for potentially sensitive store queries, schema usage, or proprietary prompts without any in-file disclosure or minimization, which is risky even if framed as telemetry.

Context-Inappropriate Capability

Medium

• Confidence: 97% confidence
• Finding: The validation script performs outbound telemetry to a remote Shopify endpoint even though its primary role is local code validation. While likely intended for product analytics, this creates an unnecessary data egress path from a developer tool and can expose sensitive validation context in environments where code, store identifiers, or model/client metadata should remain local.

Context-Inappropriate Capability

Medium

• Confidence: 95% confidence
• Finding: The script performs secondary telemetry by sending execution context and tool results to a separate endpoint unrelated to the core document-search function. In this skill context, the data sent can include user queries, model/client metadata, and returned result content, creating an unnecessary data disclosure channel without clear user notice or consent.

Context-Inappropriate Capability

Medium

• Confidence: 98% confidence
• Finding: The script sends validation data to a remote Shopify endpoint via `fetch`, including user-provided context and, in stateless mode, the full `code` content being validated. Because validation inputs may contain proprietary theme code, secrets, internal paths, or customer-specific data, this creates unintended data exfiltration risk without clear user consent or minimization.

Context-Inappropriate Capability

Medium

• Confidence: 96% confidence
• Finding: The script contains network telemetry logic (`reportValidation`) that POSTs to an external URL despite the tool's apparent purpose being local GraphQL validation. This expands the trust boundary and can leak execution metadata or user inputs off-host, which is risky in an agent skill where users may reasonably expect offline/local validation behavior.

Context-Inappropriate Capability

Medium

• Confidence: 96% confidence
• Finding: The script sends the full validated GraphQL code plus client metadata to an external endpoint via `reportValidation()`. A validation utility does not need to exfiltrate user-supplied query content to perform its core function, so this creates an unnecessary data disclosure channel that could leak proprietary queries, schema usage, store-related data, or sensitive workflow context.

Context-Inappropriate Capability

Medium

• Confidence: 96% confidence
• Finding: The script performs a documentation search but also sends telemetry to a separate endpoint containing the tool name, search query, result/error text, and client metadata. This is unrelated to the core search function and is not clearly disclosed in the script interface, creating a privacy and data-exfiltration concern because user prompts and returned content may contain sensitive internal information.

Context-Inappropriate Capability

Medium

• Confidence: 97% confidence
• Finding: The script's instrumentation path posts validation results and the full GraphQL source code to a remote endpoint via `fetch`, which can exfiltrate proprietary queries, schema usage, tokens embedded in queries, or other sensitive business logic. Because this occurs automatically during validation and the file itself provides no user-facing warning or explicit consent flow, it creates a real confidentiality risk rather than a harmless analytics event.

Intent-Code Divergence

Low

• Confidence: 91% confidence
• Finding: The instrumentation helper appears framed as generic validation reporting, but the implementation includes the full GraphQL code and client metadata in the request body. That mismatch increases the chance that users and integrators will not realize sensitive query contents are being exported, making the telemetry behavior materially more dangerous in practice.

Missing User Warnings

Medium

• Confidence: 92% confidence
• Finding: The README instructs users to run remote search and validation commands and references telemetry-related environment variables, but it does not clearly warn that code, queries, filenames, and other artifacts may be transmitted to Shopify-operated services. In an agent-skill context, this is security-relevant because users may assume validation is local and could unintentionally send proprietary source code, API queries, or customer-related data to third-party infrastructure.

Missing User Warnings

Medium

• Confidence: 95% confidence
• Finding: The script sends usage telemetry to a remote Shopify endpoint, including the user query and client/model context, without any user-facing disclosure or explicit consent in this file. Even if intended for analytics, queries may contain sensitive business, customer, or debugging information, so silent exfiltration of prompt/context data creates a real privacy and data-handling risk.

Missing User Warnings

Medium

• Confidence: 98% confidence
• Finding: The script transmits the supplied search query to a remote service over the network without any visible warning in the code path. Because the query is user-controlled and may include internal project details, customer data, or proprietary troubleshooting context, this is a genuine data exposure concern in an agent skill context.

Missing User Warnings

Medium

• Confidence: 98% confidence
• Finding: This script sends the full GraphQL validation input (`code`) plus client metadata to a remote endpoint via `reportValidation()` without any user-facing notice, consent, or minimization. Because the validated query can contain sensitive business logic, identifiers, or customer-related fields, this creates an unexpected data exfiltration channel from a local validation tool to an external service.

Missing User Warnings

Medium

• Confidence: 95% confidence
• Finding: The script sends the user's raw search query to a remote telemetry endpoint via `reportValidation(...)` after both successful and failed searches, and there is no user-facing notice or consent mechanism in this file. Search queries can contain sensitive internal code names, secrets pasted by mistake, customer data, or proprietary project details, so transmitting them off-host creates a real privacy and data-governance risk.

VirusTotal

67/67 vendors flagged this skill as clean.