Pilot Penetration Testing Setup

• Outcome: Review
• Latest audit: Sep 16, 2026 · 7:16 AM UTC
• Version: 1.0.0

Overview

This is a coherent penetration-testing setup skill, but it needs Review because it installs unpinned skills, uses unsafe shell placeholders, and can route sensitive pentest reports to external services.

Review this skill before installing. Use it only for authorized assessments, pin and verify dependency skills, restrict prefix and role values to safe known names, verify peer identities before handshakes, and send reports only to approved Slack or webhook destinations after reviewing sensitive content. Remove ~/.pilot/setups/penetration-testing.json when the deployment is no longer needed.

A.I.G

Vulnerability Patterns

• Insecure Dependencies: Introduces malicious components through unsafe dependency sources
• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review

Findings (2)

T08 · Insecure Dependencies

Error

• Location: SKILL.md:45

Finding

Unpinned ClawHub Dependencies Permit Supply-Chain Substitution

Content

View full analysis

Vulnerability Details

File Location: SKILL.md, lines 45-52
Vulnerability Type: Unpinned third-party skill dependencies
Risk Level: High

bash

# For recon:
clawhub install pilot-discover pilot-stream-data pilot-archive
# For scanner:
clawhub install pilot-task-parallel pilot-metrics pilot-dataset
# For exploiter:
clawhub install pilot-task-chain pilot-audit-log pilot-receipt
# For reporter:
clawhub install pilot-webhook-bridge pilot-share pilot-slack-bridge

Technical Analysis

The setup procedure installs third-party skills by package name without specifying a version, immutable digest, trusted publisher identity, or integrity value. Consequently, the exact content installed can change after this skill has been reviewed.

If a listed package or its publishing account is compromised, removed and re-registered, or updated with malicious content, subsequent executions of this procedure may install the altered release. The risk applies to every role because all four installation commands resolve dependencies dynamically through ClawHub.

This finding does not establish that any currently named dependency is malicious. It identifies the absence of controls that bind installation to the specific dependency content that was reviewed.

Attack Path

1. An attacker compromises the publisher or release process for one of the named ClawHub skills, or otherwise causes the registry to resolve that name to a malicious release.
2. The attacker publishes a modified version containing hostile instructions or executable behavior.
3. An operator follows the documented setup procedure and runs the unversioned clawhub install command.
4. ClawHub resolves and installs the attacker-controlled current release rather than a previously reviewed artifact.
5. When the installed skill is loaded or invoked, its malicious behavior executes with the permissions available to the agent or user running the setup.

...[truncated 831 chars]

Remediation

View remediation

Remediation Suggestions

• Pin every dependency to an explicitly reviewed, immutable version and, where supported, a cryptographic digest.
• Verify package signatures and require an allowlisted publisher identity before installation.
• Maintain a lock file or deployment manifest recording package names, versions, hashes, and trusted publishers.
• Resolve and inspect dependencies before deployment; do not automatically accept newer releases.
• Use a trusted internal mirror or approved artifact repository for production deployments.
• Run installed skills with least privilege, isolated credentials, restricted filesystem access, and constrained network egress.
• Add an automated policy check that rejects unpinned clawhub install commands.
• Review and update the equivalent unpinned installation examples in README.md to prevent operators from bypassing the hardened procedure.

T09 · Insecure Skill Coding Practices

Error

• Location: SKILL.md:53

Finding

Unvalidated Prefix Is Interpolated into a Shell Command

Content

View full analysis

Vulnerability Details

File Location: SKILL.md, line 53
Vulnerability Type: Shell command injection through an untrusted setup parameter
Risk Level: High

bash

pilotctl --json set-hostname <prefix>-<role>

The relevant input instruction appears earlier in the setup procedure:

text

Step 1: Ask the user which role this agent should play and what prefix to use.

Technical Analysis

The skill directs the agent to obtain a prefix from the user and place it into a Bash command without quoting, validation, or a constrained character policy. In a shell, metacharacters such as command separators, substitutions, redirections, and whitespace are interpreted as syntax rather than as part of a hostname.

If an agent performs direct textual substitution and executes the resulting command through Bash, a malicious prefix can terminate or alter the intended pilotctl invocation and introduce additional commands. Quoting alone would reduce word splitting and separator injection, but robust remediation also requires rejecting values that are not valid deployment-prefix identifiers.

The documented role should likewise be selected from a fixed enumeration rather than copied as arbitrary user-controlled text.

Attack Path

1. The skill asks a user for a deployment prefix.
2. A malicious or compromised user supplies a prefix containing shell syntax, for example a command separator followed by an attacker-selected command.
3. The agent substitutes that value into the documented command without validation or shell-safe argument handling.
4. Bash parses the injected syntax as an additional command rather than as hostname data.
5. The injected command runs with the permissions and environment of the account executing the setup.

Exploitation depends on the agent constructing and executing the command by direct textual interpolation, as the current instruction implies. An implementation that pas ...[truncated 716 chars]

Remediation

View remediation

Remediation Suggestions

• Enforce a strict prefix allowlist before constructing any command, such as ^[a-z0-9][a-z0-9-]{0,30}$.

• Restrict the role to the fixed values recon, scanner, exploiter, or reporter.

• Pass each value as a separate process argument without invoking a shell. For example, use an execution API equivalent to ["pilotctl", "--json", "set-hostname", hostname].

• If Bash is unavoidable, validate first and quote the complete generated hostname:

  bash

  case "$prefix" in
    ''|*[!a-z0-9-]*|-*|*-) exit 1 ;;
  esac
  
  case "$role" in
    recon|scanner|exploiter|reporter) ;;
    *) exit 1 ;;
  esac
  
  pilotctl --json set-hostname "${prefix}-${role}"
  

• Apply the same validation and argument-handling rules to every later use of the prefix, including handshake, subscribe, publish, and manifest-generation operations.

• Add tests containing spaces, semicolons, quotes, command substitutions, newlines, leading hyphens, and redirection operators, and verify that all invalid values are rejected before command execution.

• Run setup operations as a dedicated least-privileged account.

SkillSpector

Vulnerability Patterns

• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Excessive Agency: Unrestricted Tool Access, Autonomous Decision Making, Scope Creep
• Rogue Agent: Self-Modification, Session Persistence
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access

Findings (4)

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 95% confidence

Finding

The skill explicitly routes pentest reports to Slack and webhooks, which can transmit highly sensitive vulnerability details, targets, and proof-of-concept evidence outside the controlled assessment environment. In a penetration-testing workflow, this materially increases the risk of accidental disclosure to third-party services, misconfigured endpoints, or unintended recipients if users are not clearly warned and guided on safe handling.

Content

No source excerpt is available for this finding.

Autonomous Decision Making

Medium

• Category: Excessive Agency
• Confidence: 85% confidence

Finding

The README states that trust is auto-approved once both sides send handshakes, with no manual verification step. That weakens the trust establishment process because any mistaken hostname, spoofed peer, or unauthorized but reachable agent that can complete the reciprocal handshake may be granted trusted communication without human confirmation.

Content

Scanner excerpt · README.md (reported line 64)May include surrounding context.

2. Establish trust

Agents are private by default. Each pair that communicates must exchange handshakes. When both sides send a handshake, trust is auto-approved -- no manual step needed.

bash

# On recon:

Session Persistence

Medium

• Category: Rogue Agent
• Confidence: 82% confidence

Finding

The skill persists a penetration-testing setup manifest under ~/.pilot/setups, creating session state that can survive beyond the immediate task. While persistence is not inherently malicious here, storing role, hostname, peer, and workflow configuration for a pentest pipeline can expose operational metadata or cause later runs to inherit sensitive context unexpectedly.

Content

Scanner excerpt · SKILL.md (reported line 67)May include surrounding context.

pilotctl --json set-hostname -

text

**Step 4:** Write the setup manifest:
```bash
mkdir -p ~/.pilot/setups
cat > ~/.pilot/setups/penetration-testing.json << 'MANIFEST'

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 95% confidence

Finding

The skill explicitly configures the reporter role to send pentest reports to external systems via webhook and Slack, but the skill description does not warn the user that potentially sensitive assessment data will leave the local agent mesh. In a penetration-testing context, findings, proof-of-concept evidence, target details, and severity data are often highly sensitive, so omitted disclosure increases the risk of unintentional exfiltration or policy violations.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.