Back to skill

Security audit

qa-suite

Security checks for vulnerabilities and agentic risk

Overview

This testing skill needs review because it can automatically run data-changing tests without clearly restricting execution to approved, isolated targets.

Before installing, restrict execution to explicitly approved local or staging targets with disposable data and dedicated test accounts. Require separate authorization after planning, bind test services to loopback, and review Codecov uploads before enabling them. Independently check coverage and failure evidence rather than relying on the bundled gate's verdict. The clean static scan does not resolve these verified workflow concerns.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Vulnerability Patterns
  • Output HandlingUnvalidated Output Injection, Cross-Context Output, Unbounded Output
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (11)

Description-Behavior Mismatch

High
Category
MCP Tool Poisoning
Confidence
90% confidence
Finding

The integration-test command and frontend dependency installation fit the declared purpose. However, cargo test is explicitly labeled as backend unit testing and runs unit tests directly, contradicting the stated delegation boundary. The absence of implemented E2E steps in this partial chunk is not itself a mismatch.

Content

No source excerpt is available for this finding.

Skill declares no tool scope ('permissions' or 'allowed-tools') but code capabilities were detected: file_read.

Medium
Category
MCP Least Privilege
Confidence
70% confidence
Finding

Without declared permissions the skill's intent is opaque and cannot be validated.

Content

No source excerpt is available for this finding.

Real-service integration tests are executed without warning about potential changes to service data or external side effects.

Medium
Category
Output Handling
Confidence
86% confidence
Finding

The workflow directs the agent to write and run tests against real services, including failure and data-integrity scenarios, without explicitly requiring disposable targets or checking that configured endpoints and credentials are non-production. Docker and Testcontainers preferences reduce the risk, but do not prevent an existing configuration or containerized application from reaching shared databases or external services, potentially modifying data or triggering external actions.

Content

No source excerpt is available for this finding.

The CI workflow uploads coverage data to Codecov without warning about third-party disclosure.

Medium
Category
Output Handling
Confidence
91% confidence
Finding

The workflow invokes Codecov with a coverage report, causing repository-derived coverage data to be transmitted to an external service. Unlike a tool-catalog listing, this is an executable integration example, but the document does not explain the external upload or its privacy implications; the same omission appears at lines 130–133.

Content

No source excerpt is available for this finding.

MCP server referenced without pinned version: 'npx vitest'.

Medium
Category
MCP Rug Pull
Confidence
70% confidence
Finding

npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.

Content

No source excerpt is available for this finding.

Mandatory adversarial tests lack a warning to use an authorized, isolated test environment.

Medium
Category
Output Handling
Confidence
88% confidence
Finding

The skill mandates injection, repeated-submission, and cross-user authorization tests without requiring an authorized target or isolated test environment. If generated tests or live browser operations target a production system, these actions could modify real data, trigger account lockouts, or access another user's information. The legitimate QA context, default code-generation mode, and mocked failure examples reduce immediate risk, but do not constrain where tests are ultimately executed.

Content

No source excerpt is available for this finding.

The Docker setup exposes test databases on host network interfaces without warning the user.

Medium
Category
Output Handling
Confidence
98% confidence
Finding

The Compose port mappings publish PostgreSQL and Redis on all host interfaces by default, potentially allowing access from other machines where host firewall and network rules permit. PostgreSQL uses credentials disclosed in the example, while Redis has no authentication configured, allowing reachable attackers to read or modify test data or disrupt services. The test-only context limits expected data sensitivity, but the setup guidance does not warn about exposure on shared development or CI hosts.

Content

No source excerpt is available for this finding.

Having a user-facing UI is an overly broad activation condition.

Medium
Category
Trigger Abuse
Confidence
97% confidence
Finding

The activation list includes any project with a web, mobile, or desktop UI, without requiring a testing request. This can activate the skill during unrelated UI development or maintenance, and the file provides no exclusion conditions.

Content

No source excerpt is available for this finding.

Automatic execution of comprehensive UI tests lacks a warning about effects on application data.

Medium
Category
Output Handling
Confidence
96% confidence
Finding

The file directs the agent to execute tests immediately after generating a plan, even when the user's request may only concern planning. Its exhaustive coverage includes state-changing controls such as Delete and Save, repeated submissions, and injection inputs, but it does not require an isolated environment or authorization for destructive operations; against a live application, this could modify data or trigger external side effects.

Content

No source excerpt is available for this finding.

The documented mandatory planning-integrity gate permits reports with no coverage matrix.

Medium
Category
Excessive Agency
Confidence
99% confidence
Finding

The validator skips planning-integrity checks when coverage_matrix is absent or null; empty matrices and unsupported matrix types also produce no violation. A report with matching positive planned/executed counts and no failures can therefore receive PASSED without demonstrating coverage, undermining the skill's mandatory QA gate and potentially allowing untested functionality through downstream release checks.

Content

No source excerpt is available for this finding.

The documented mandatory evidence gate does not require screenshots and treats missing error evidence as advisory.

Medium
Category
Excessive Agency
Confidence
99% confidence
Finding

The documented evidence gate requires both error details and a screenshot for every failure, but the implementation never validates screenshots and only warns when error details are absent. Reports containing P2/P3 failures can consequently receive CONDITIONAL_PASS without the required evidence; in this QA execution skill, that weakens defect verification and permits acknowledged progression based on incomplete results rather than enforcing the promised gate.

Content

No source excerpt is available for this finding.