pilot-leader-election
Implement leader election protocols with automatic failover detection.
Commands
Announce candidacy
pilotctl --json publish "registry-hostname" "election:$ELECTION_GROUP" \
--data "{\"type\":\"election\",\"candidate\":\"$AGENT_ID\",\"priority\":$PRIORITY,\"term\":$TERM}"
Declare victory as leader
pilotctl --json publish "registry-hostname" "election:$ELECTION_GROUP" \
--data "{\"type\":\"leader\",\"leader\":\"$AGENT_ID\",\"term\":$TERM,\"elected_at\":\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\"}"
Send leader heartbeat
pilotctl --json publish "registry-hostname" "election:$ELECTION_GROUP" \
--data "{\"type\":\"heartbeat\",\"leader\":\"$AGENT_ID\",\"term\":$TERM,\"timestamp\":\"$(date -u +%s)\"}"
Detect leader failure
LAST_HEARTBEAT=$(pilotctl --json inbox \
| jq -r '[.messages[] | select(.topic == "election:'$ELECTION_GROUP'" and .payload.type == "heartbeat")] | sort_by(.payload.timestamp) | last | .payload.timestamp')
ELAPSED=$(( $(date -u +%s) - LAST_HEARTBEAT ))
if [ "$ELAPSED" -gt 10 ]; then
echo "Leader timeout, starting election"
fi
Workflow Example
Bully algorithm with priority-based election:
#!/bin/bash
ELECTION_GROUP="task-coordinator"
AGENT_ID=$(pilotctl --json info | jq -r '.node_id')
PRIORITY=$(echo -n "$AGENT_ID" | cksum | cut -d' ' -f1)
REGISTRY_HOST="registry.example.com"
# Announce candidacy
pilotctl --json publish "$REGISTRY_HOST" "election:$ELECTION_GROUP" \
--data "{\"type\":\"election\",\"candidate\":\"$AGENT_ID\",\"priority\":$PRIORITY,\"term\":$TERM}"
# Wait and check if highest priority
sleep 10
HIGHEST=$(pilotctl --json inbox \
| jq -r '[.messages[] | select(.topic == "election:'$ELECTION_GROUP'" and .payload.type == "election")] | sort_by(.payload.priority) | last | .payload.candidate')
if [ "$HIGHEST" = "$AGENT_ID" ]; then
pilotctl --json publish "$REGISTRY_HOST" "election:$ELECTION_GROUP" \
--data "{\"type\":\"leader\",\"leader\":\"$AGENT_ID\",\"term\":$TERM}"
fi
Dependencies
Requires pilot-protocol skill, jq, and cksum.