Pilot Knowledge Base Rag Setup

Deploy a knowledge base RAG pipeline with 4 agents. Use this skill when: 1. User wants to set up a document ingestion and retrieval pipeline 2. User is confi...

Security Scan

Instruction-only skill that describes setting up a 4-agent RAG pipeline and only requires the two CLI tools it uses; its declared requirements and runtime instructions are consistent with its stated purpose.

Purpose & Capability

The name and description match the instructions: the SKILL.md uses pilotctl and clawhub to install skills, set hostnames, exchange handshakes, and publish messages between agents. The declared required binaries (pilotctl, clawhub) are exactly the tools referenced.

Instruction Scope

Runtime instructions are limited to installing listed pilot sub-skills, writing a manifest to ~/.pilot/setups/knowledge-base-rag.json, running pilotctl commands (handshake, publish, send-file, task submit), and configuring hostnames — all within the scope of provisioning and testing a RAG pipeline. There are no directives to read unrelated system files or to transmit data to unexpected external endpoints.

Install Mechanism

This is an instruction-only skill with no install spec or remote downloads. That is the lowest-risk setup; it relies on the system already having pilotctl and clawhub available.

Credentials

The skill itself requests no environment variables or credentials, which is proportionate. Note: some of the sub-skills it installs (e.g., pilot-s3-bridge, pilot-database-bridge) will likely require cloud or database credentials at runtime; those are not requested here and should be provisioned/verified separately by the user.

Persistence & Privilege

always is false and the skill does not request any special platform-wide privileges. It suggests writing a manifest under the user's home config (~/.pilot), which is reasonable for per-deployment state and does not modify other skills' configs.

Assessment

This skill appears coherent and limited to orchestrating pilot agents. Before installing: (1) ensure pilotctl and clawhub are from trusted sources; (2) review and understand the sub-skills (pilot-s3-bridge, pilot-database-bridge, etc.) because those components may require cloud or DB credentials and will affect network access; (3) confirm port and network firewall rules for the pipeline (port 1001 is used in examples); and (4) remember the README/AGPL license and back up existing ~/.pilot config if you want to revert. If you plan to use S3 or an external DB, supply credentials only to the specific sub-skill that needs them and verify where those credentials are stored.

Knowledge Base (RAG) Setup

Deploy 4 agents: ingest, embed, index, and query.

Roles

Role	Hostname	Skills	Purpose
ingest	<prefix>-rag-ingest	pilot-s3-bridge, pilot-share, pilot-chunk-transfer, pilot-cron	Pulls and chunks documents
embedder	<prefix>-rag-embedder	pilot-task-parallel, pilot-share, pilot-metrics, pilot-task-chain	Generates vector embeddings
indexer	<prefix>-rag-indexer	pilot-database-bridge, pilot-share, pilot-task-chain, pilot-health	Stores embeddings in vector DB
query	<prefix>-rag-query	pilot-api-gateway, pilot-health, pilot-load-balancer, pilot-metrics	Serves search queries

Setup Procedure

Step 1: Ask the user which role and prefix.

Step 2: Install skills:

# ingest:
clawhub install pilot-s3-bridge pilot-share pilot-chunk-transfer pilot-cron
# embedder:
clawhub install pilot-task-parallel pilot-share pilot-metrics pilot-task-chain
# indexer:
clawhub install pilot-database-bridge pilot-share pilot-task-chain pilot-health
# query:
clawhub install pilot-api-gateway pilot-health pilot-load-balancer pilot-metrics

Step 3: Set hostname and write manifest to ~/.pilot/setups/knowledge-base-rag.json.

Step 4: Handshake along the pipeline: ingest↔embedder, embedder↔indexer, indexer↔query.

Manifest Templates Per Role

ingest

{
  "setup": "knowledge-base-rag", "role": "ingest", "role_name": "Document Ingestion",
  "hostname": "<prefix>-rag-ingest",
  "skills": {
    "pilot-s3-bridge": "Pull documents from S3 buckets.",
    "pilot-share": "Send document files to embedder.",
    "pilot-chunk-transfer": "Split large documents into chunks.",
    "pilot-cron": "Schedule periodic ingestion sweeps."
  },
  "data_flows": [{ "direction": "send", "peer": "<prefix>-rag-embedder", "port": 1001, "topic": "doc-ingested", "description": "Document chunks" }],
  "handshakes_needed": ["<prefix>-rag-embedder"]
}

embedder

{
  "setup": "knowledge-base-rag", "role": "embedder", "role_name": "Embedding Generator",
  "hostname": "<prefix>-rag-embedder",
  "skills": {
    "pilot-task-parallel": "Generate embeddings in parallel for throughput.",
    "pilot-share": "Receive docs from ingest, send embeddings to indexer.",
    "pilot-metrics": "Track embedding throughput and latency.",
    "pilot-task-chain": "Chain chunking and embedding steps."
  },
  "data_flows": [
    { "direction": "receive", "peer": "<prefix>-rag-ingest", "port": 1001, "topic": "doc-ingested", "description": "Document chunks" },
    { "direction": "send", "peer": "<prefix>-rag-indexer", "port": 1001, "topic": "embeddings-ready", "description": "Vector embeddings" }
  ],
  "handshakes_needed": ["<prefix>-rag-ingest", "<prefix>-rag-indexer"]
}

indexer

{
  "setup": "knowledge-base-rag", "role": "indexer", "role_name": "Vector Indexer",
  "hostname": "<prefix>-rag-indexer",
  "skills": {
    "pilot-database-bridge": "Write embeddings to vector database.",
    "pilot-share": "Receive embeddings from embedder.",
    "pilot-task-chain": "Chain indexing operations.",
    "pilot-health": "Monitor index health and query latency."
  },
  "data_flows": [
    { "direction": "receive", "peer": "<prefix>-rag-embedder", "port": 1001, "topic": "embeddings-ready", "description": "Vector embeddings" },
    { "direction": "receive", "peer": "<prefix>-rag-query", "port": 1001, "topic": "search-query", "description": "Search queries" },
    { "direction": "send", "peer": "<prefix>-rag-query", "port": 1001, "topic": "search-results", "description": "Ranked results" }
  ],
  "handshakes_needed": ["<prefix>-rag-embedder", "<prefix>-rag-query"]
}

query

{
  "setup": "knowledge-base-rag", "role": "query", "role_name": "Query Server",
  "hostname": "<prefix>-rag-query",
  "skills": {
    "pilot-api-gateway": "Accept search queries from external clients.",
    "pilot-health": "Monitor query endpoint health.",
    "pilot-load-balancer": "Distribute queries across indexer replicas.",
    "pilot-metrics": "Track QPS, latency, result quality."
  },
  "data_flows": [
    { "direction": "send", "peer": "<prefix>-rag-indexer", "port": 1001, "topic": "search-query", "description": "Search queries" },
    { "direction": "receive", "peer": "<prefix>-rag-indexer", "port": 1001, "topic": "search-results", "description": "Ranked results" }
  ],
  "handshakes_needed": ["<prefix>-rag-indexer"]
}

Data Flows

• ingest → embedder : document chunks (port 1001)
• embedder → indexer : vector embeddings (port 1001)
• query ↔ indexer : search queries and results (port 1001)

Workflow Example

# On ingest:
pilotctl --json send-file <prefix>-rag-embedder ./docs/guide.pdf
pilotctl --json publish <prefix>-rag-embedder doc-ingested '{"doc_id":"doc-42","chunks":24}'
# On embedder:
pilotctl --json publish <prefix>-rag-indexer embeddings-ready '{"doc_id":"doc-42","vectors":24,"dims":1536}'
# On query:
pilotctl --json task submit <prefix>-rag-indexer --task '{"query":"How does auth work?","top_k":5}'

Dependencies

Requires pilot-protocol skill, pilotctl binary, clawhub binary, and a running daemon.