ClawHub
Back to skill

Security audit

Pilot Data Labeling Pipeline Setup

Security checks for vulnerabilities and agentic risk

Overview

This looks like a legitimate data-labeling setup skill, but it warrants Review because it installs unpinned add-on skills, creates persistent agent configuration, and enables trusted data flows including external export notifications.

Review this skill before installing in production. Use only trusted, pinned versions of the dependent skills, verify peer hostnames or identities before handshakes, confirm S3/shared-storage and webhook destinations, and remove or inspect ~/.pilot/setups/data-labeling-pipeline.json when the setup is no longer needed.

A.I.G

Vulnerability Patterns
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T08 · Insecure Dependencies

Warning
Location
SKILL.md:43
Finding

Unpinned Third-Party Skill Installations Create a Supply-Chain Risk

Content
View full analysis
-ingester # On server 2 (auto labeling) clawhub install pilot-task-router pilot-dataset pilot-metrics pilotctl set-hostname -labeler # On server 3 (quality review) clawhub install pilot-review pilot-event-filter pilot-alert pilotctl set-hostname -reviewer # On server 4 (dataset export) clawhub install pilot-dataset pilot-share pilot-webhook-bridge pilotctl set-hostname -exporter ``` ### Technical Analysis The setup installs multiple executable third-party Skills using mutable package names without specifying exact versions, integrity hashes, trusted publisher identities, or signature-verification requirements. The audited artifact does not contain the implementations of these dependencies, so their effective behavior cannot be reviewed from this project. Because `SKILL.md` permits Bash and directs the agent to install these packages, a compromised or unexpectedly changed dependency could execute with the permissions available to the agent. No evidence establishes that any currently named package is malicious; the vulnerability is the absence of controls that guarantee the reviewed dependency is the dependency eventually installed. ### Attack Path 1. An attacker compromises th ...[truncated 1705 chars]
Remediation
View remediation

SkillSpector

Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Rogue AgentSelf-Modification, Session Persistence
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Findings (7)

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

This markdown file describes a pipeline that ingests raw data, processes images/text/audio, and exports training-ready datasets, which can involve sensitive user data. The document provides operational steps for publishing and exporting data but does not include any warning about handling sensitive content, privacy, or ensuring authorization before transmitting or exporting datasets.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
86% confidence
Finding

The exporter role and data-flow section explicitly state that completed datasets are published to storage and that notifications are sent to downstream external consumers. Because this markdown serves as the usage guide, it should warn users that these actions can transmit potentially sensitive metadata or dataset contents outside the local system.

Content

No source excerpt is available for this finding.

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
93% confidence
Finding

The README states that trust is automatically approved once both sides exchange handshakes, with no manual verification step. If an operator handshakes the wrong hostname, or naming is spoofed or misconfigured, agents may establish trust with an unintended peer and permit data exchange across the pipeline.

Content

Scanner excerpt · README.md (reported line 65)May include surrounding context.

2. Establish trust

Agents are private by default. Each pair that communicates must exchange handshakes. When both sides send a handshake, trust is auto-approved -- no manual step needed.

bash
# On ingester:

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
86% confidence
Finding

The skill describes multiple inter-agent transmissions and an external notification path but does not clearly warn that dataset metadata, item identifiers, hostnames, and event topics will be sent across peers or outward-facing endpoints. In a data labeling context, such metadata can itself be sensitive and can increase exposure if users assume the skill is only doing local setup.

Content

No source excerpt is available for this finding.

Session Persistence

Medium
Category
Rogue Agent
Confidence
93% confidence
Finding

The manifest write establishes persistent session state in ~/.pilot/setups, causing the agent's configuration to survive beyond the immediate task. Persistence is security-relevant because later runs may automatically rely on stored peers, roles, and communication patterns that the user no longer expects or that were created with overly broad settings.

Content

Scanner excerpt · SKILL.md (reported line 67)May include surrounding context.

pilotctl --json set-hostname -

text

**Step 4:** Write the setup manifest:
```bash
mkdir -p ~/.pilot/setups
cat > ~/.pilot/setups/data-labeling-pipeline.json << 'MANIFEST'

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The skill instructs the agent to create and overwrite a manifest under ~/.pilot/setups, which is a persistent local configuration change. Writing files in a user's home directory without a clear warning, preview, or consent can silently alter future agent behavior and may persist unsafe peer, flow, or hostname settings beyond the current session.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
83% confidence
Finding

The exporter role is explicitly configured to publish datasets to shared storage and send webhook-style notifications to external consumers. In a setup skill, this expands the trust boundary beyond the internal labeling pipeline and can cause unintentional data disclosure if users deploy it without understanding that labeled datasets and related metadata may leave the local agent mesh.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.