Back to skill

Security audit

Pilot Knowledge Base Rag Setup

Security checks for vulnerabilities and agentic risk

Overview

This RAG setup is coherent, but it asks agents to install multiple unpinned integration skills and auto-establish trust between data-handling nodes without enough scoping or verification guidance.

Review this before installing in any environment with private documents, production S3 buckets, vector databases, or public query endpoints. Pin and verify every dependent skill, validate the deployment prefix, confirm peer identities before handshakes, restrict each role's credentials to only what it needs, and set retention and audit controls for documents, embeddings, queries, and results.

Vulnerability Patterns
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (2)

T08 · Insecure Dependencies

Error
Location
SKILL.md:48
Finding

Unpinned Third-Party Skill Installation Creates a Supply-Chain Risk

Content
View full analysis
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Warning
Location
README.md:41
Finding

User-Controlled Deployment Prefix Is Inserted into Shell Command Templates Without Validation

Content
View full analysis
-rag-ingest # On embedding node (GPU recommended) clawhub install pilot-task-parallel pilot-share pilot-metrics pilot-task-chain pilotctl set-hostname -rag-embedder # On indexer node clawhub install pilot-database-bridge pilot-share pilot-task-chain pilot-health pilotctl set-hostname -rag-indexer # On query server clawhub install pilot-api-gateway pilot-health pilot-load-balancer pilot-metrics pilotctl set-hostname -rag-query ``` ### Technical Analysis The documentation tells the user to replace `` with a deployment-specific value, while the Skill instructs the Agent to ask the user for a role and prefix. That value is subsequently placed into Bash command templates without a defined validation rule or shell-safe argument handling. If an Agent implements the instructions through direct textual substitution, shell metacharacters contained in a malicious prefix could be interpreted by Bash rather than treated solely as hostname data. Quoting alone would reduce some risks but would not provide an adequate hostname policy; strict allowlist validation is required. The repository does not contain an executable setup script, so exploitation depends on an Agent or user constructing and executing commands through unsafe textual substitution. Nevertheless, the documented workflow fails to establish the validation boundary needed to use user-controlled data safely in Bash. ### Attack Path 1. The setup Agent asks a user to provide a deployment prefix as directed by the Skill. 2. A malicious user supplies a prefix containing shell control syntax or command su ...[truncated 1191 chars]
Remediation
View remediation
Vulnerability Patterns
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Rogue AgentSelf-Modification, Session Persistence
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Findings (4)

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
89% confidence
Finding

The README instructs users to ingest documents from S3 and shared directories and forward them between multiple agents, but it provides no warning about handling confidential, regulated, or proprietary data. In a RAG pipeline, documents and derived metadata may traverse several services and storage layers, increasing the chance of unintended exposure, over-collection, or insecure retention if operators assume the setup is safe by default.

Content

No source excerpt is available for this finding.

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
85% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · README.md (reported line 64)May include surrounding context.

2. Establish trust

Agents are private by default. Each pair that communicates must exchange handshakes. When both sides send a handshake, trust is auto-approved -- no manual step needed.

bash
# ingest <-> embedder

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The setup tells operators that mutual handshakes result in trust being auto-approved with no manual verification step, but it does not warn about the security implications of establishing bidirectional trust. If a hostname is mistyped, spoofed, or an unintended peer is contacted, this could authorize document transfer, embedding submission, indexing, or query access between the wrong agents and widen lateral movement opportunities.

Content

No source excerpt is available for this finding.

Session Persistence

Medium
Category
Rogue Agent
Confidence
60% confidence
Finding

Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.

Content

Scanner excerpt · SKILL.md (reported line 62)May include surrounding context.

clawhub install pilot-api-gateway pilot-health pilot-load-balancer pilot-metrics

text

**Step 3:** Set hostname and write manifest to `~/.pilot/setups/knowledge-base-rag.json`.

**Step 4:** Handshake along the pipeline: ingest↔embedder, embedder↔indexer, indexer↔query.

Static analysis

No suspicious patterns detected.