Pilot Dataset
Exchange structured datasets with schema negotiation and metadata over Pilot Protocol. Use this skill when: 1. You need to share CSV, JSON, or Parquet datasets with schema information 2. You want to negotiate data formats and transformations before transfer 3. You need to maintain dataset lineage and provenance metadata Do NOT use this skill when: - You need to transfer unstructured files (use pilot-share instead) - You need real-time data streaming (use pilot-stream-data instead) - You need ML model files (use pilot-model-share instead)
pilot-dataset
Structured dataset exchange with schema negotiation, format conversion, and provenance tracking.
Commands
Publish Dataset Availability
pilotctl --json publish "$PEER" datasets --data '{"type":"dataset_available","name":"sales_data","format":"csv","rows":1000}'
Request Dataset
pilotctl --json send-message "$DEST" --data '{"type":"dataset_request","name":"sales_data","preferred_format":"json"}'
Send Dataset with Metadata
pilotctl --json send-message "$DEST" --data '{"type":"dataset_metadata","name":"sales_data","schema":{"columns":["date","amount"]}}'
pilotctl --json send-file "$DEST" "$DATASET_FILE"
Validate Schema
EXPECTED="date,amount,customer_id"
ACTUAL=$(head -1 "$DATASET_FILE")
[ "$ACTUAL" = "$EXPECTED" ] && echo "Schema validated"
Workflow Example
#!/bin/bash
# Dataset exchange
PEER="agent-b"
publish_dataset() {
local file="$1"
local name="${2:-$(basename $file .csv)}"
local rows=$(wc -l < "$file")
pilotctl --json publish "$PEER" datasets \
--data "{\"type\":\"dataset_available\",\"name\":\"$name\",\"format\":\"csv\",\"rows\":$rows}"
}
request_dataset() {
local name="$1"
local publisher="$2"
pilotctl --json send-message "$publisher" \
--data "{\"type\":\"dataset_request\",\"name\":\"$name\",\"preferred_format\":\"csv\"}"
sleep 2
pilotctl --json received
}
publish_dataset "data.csv" "my-dataset"
Dependencies
Requires pilot-protocol, pilotctl, jq, and optionally python3 for format conversion.
Related skills
- Downloads
- 838All time
- Security audit
- Pass
- Last updated
- 6mo ago
- Current version
- v1.0.0
- License
- MIT-0
