Pilot Quality Assurance Pipeline Setup

• Outcome: Pass
• Latest audit: Sep 16, 2026 · 7:01 AM UTC
• Version: 1.0.0

Overview

This is a coherent QA pipeline setup skill, but users should verify dependencies, peers, and outbound report contents before using it.

Install only after confirming the referenced Pilot skills come from trusted publishers and fixed versions. Use a simple safe prefix such as lowercase letters, numbers, and hyphens only. Before enabling reporter integrations, restrict webhook, Slack, dashboard, and bug-tracker destinations and redact secrets, customer data, screenshots, and logs as needed. Verify peer identities before accepting handshakes.

A.I.G

Vulnerability Patterns

• Insecure Dependencies: Introduces malicious components through unsafe dependency sources
• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review

Findings (2)

T08 · Insecure Dependencies

Warning

• Location: SKILL.md:48

Finding

Unpinned Third-Party Skill Installation Creates a Supply-Chain Risk

Content

View full analysis

-test-generator # On server 2 (test executor) clawhub install pilot-task-parallel pilot-share pilot-metrics pilotctl set-hostname -executor # On server 3 (QA reporter) clawhub install pilot-webhook-bridge pilot-alert pilot-slack-bridge pilotctl set-hostname -reporter ``` ### Technical Analysis The setup procedure installs nine external skills using mutable package names. It does not specify immutable versions or content digests, verify package signatures or publisher identities, or constrain resolution to a trusted registry. Consequently, the code reviewed in this project does not fully define the code that will execute after setup. A package owner could publish a compromised later release, a registry account could be taken over, or an unsafe package source could resolve one of the names to attacker-controlled content. This is particularly sensitive for the reporter dependencies because their documented responsibilities include sending information to webhooks, issue trackers, dashboards, and Slack. ### Attack Path 1. An attacker compromises the publisher account, registry entry, or distribution infrastructure associated with one of the named skills. 2. The attacker publishes or substitutes a malicious package under the expected mutable name. 3. An operator follows the documented setup procedure and runs `clawhub install` without a version ...[truncated 1009 chars]

Remediation

View remediation

T09 · Insecure Skill Coding Practices

Warning

• Location: SKILL.md:45

Finding

Unvalidated Deployment Prefix Can Enable Shell Command Injection

Content

View full analysis

- ``` ``` `SKILL.md:103-108`: ```bash # test-generator <-> executor: pilotctl --json handshake -executor "setup: quality-assurance-pipeline" pilotctl --json handshake -test-generator "setup: quality-assurance-pipeline" # executor <-> reporter: pilotctl --json handshake -reporter "setup: quality-assurance-pipeline" pilotctl --json handshake -executor "setup: quality-assurance-pipeline" ``` `README.md:35-50`: ```markdown Replace `` with a unique name for your deployment (e.g. `acme`). ### 1. Install skills on each server ```bash # On server 1 (test generator) clawhub install pilot-task-router pilot-dataset pilot-cron pilotctl set-hostname -test-generator # On server 2 (test executor) clawhub install pilot-task-parallel pilot-share pilot-metrics pilotctl set-hostname -executor # On server 3 (QA reporter) clawhub install pilot-webhook-bridge pilot-alert pilot-slack-bridge pilotctl set-hostname -reporter ``` ``` ### Technical Analysis The skill explicitly asks the user to provide a deployment prefix and then directs the agent to substitute that value into Bash commands. No permitted-character policy, length restriction, hostname validation, or shell-safe argumen ...[truncated 1984 chars]

Remediation

View remediation

&2 exit 1 ;; esac ``` 3. Construct the final hostname only after validation and validate the complete value against hostname length requirements. 4. Quote every shell argument: ```bash hostname="${prefix}-${role}" pilotctl --json set-hostname "$hostname" ``` 5. Prefer structured process execution that passes an argument array directly to `pilotctl` without invoking a shell. 6. Never use `eval`, shell command construction, or command substitution to process the prefix. 7. Apply the same validated hostname variable to handshake, subscribe, and publish operations rather than repeatedly interpolating raw user input. 8. Document the accepted prefix grammar explicitly so operators and agents reject unsafe input before executing any setup command. ]]>

SkillSpector

Vulnerability Patterns

• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Excessive Agency: Unrestricted Tool Access, Autonomous Decision Making, Scope Creep
• Rogue Agent: Self-Modification, Session Persistence
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access

Findings (4)

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 89% confidence

Finding

The README instructs the reporter to send QA reports, bug data, logs, and screenshots to external dashboards, bug trackers, and Slack, but it provides no guidance on sanitizing or minimizing test artifacts first. Test results often contain stack traces, environment details, screenshots, and sometimes secrets or PII, so this workflow can unintentionally exfiltrate sensitive internal data to third-party systems.

Content

No source excerpt is available for this finding.

Autonomous Decision Making

Medium

• Category: Excessive Agency
• Confidence: 95% confidence

Finding

The trust model auto-approves communication once both sides exchange handshakes, with no documented human verification, identity validation, or out-of-band confirmation. If an attacker can spoof naming, misdirect setup, or gain temporary access to one node, they may establish trusted channels and inject tasks, receive artifacts, or pivot across the QA pipeline.

Content

Scanner excerpt · README.md (reported line 54)May include surrounding context.

2. Establish trust

Agents are private by default. Each pair that communicates must exchange handshakes. When both sides send a handshake, trust is auto-approved -- no manual step needed.

bash

# test-generator <-> executor

Session Persistence

Medium

• Category: Rogue Agent
• Confidence: 60% confidence

Finding

Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.

Content

Scanner excerpt · SKILL.md (reported line 66)May include surrounding context.

pilotctl --json set-hostname -

text

**Step 4:** Write the setup manifest:
```bash
mkdir -p ~/.pilot/setups
cat > ~/.pilot/setups/quality-assurance-pipeline.json << 'MANIFEST'

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 93% confidence

Finding

The skill explicitly configures the reporter role to send QA reports to external destinations over webhook/Slack/bug trackers, but the setup guidance does not clearly warn the user that test results, logs, screenshots, and related QA artifacts may leave the local environment. In a QA pipeline, those artifacts can contain sensitive data such as internal URLs, stack traces, credentials in logs, or customer test data, so omission of a disclosure meaningfully increases the risk of unintended data exfiltration.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.