AI Cluster Pre-flight Check

Pre-flight check for GPU cluster nodes — node validation before training, check cluster node health, is my GPU node ready. 26 health checks covering GPU, PCI...

Security Scan

The skill's code broadly matches its stated purpose (node pre‑flight checks) but there are several mismatches and network/execution behaviors you should understand before installing or running it.

Purpose & Capability

The name/description match the included scripts: the code implements ~26 local and cross‑node hardware and config checks (GPU, PCIe, RDMA, NUMA, firewall, BIOS, switch, etc.). Required binaries (GPU vendor tools) are appropriate. However the manifest under‑declares other binaries the scripts use (ssh, ip, lspci, ethtool, setpci, dmidecode, ipmitool, docker, etc.), and declares jq as a required binary although the scripts implement JSON formatting without jq. Also the registry marks PREFLIGHT_NODE_ID as the "primary credential" despite it being a harmless node identifier.

Instruction Scope

Runtime instructions (run preflight.sh) are explicit and the scripts do many privileged/local inspections: reading /proc/cmdline, /sys entries, lspci, lsmod, setpci, dmidecode, ipmitool, ip/ethtool, and more. Cross‑node checks will attempt SSH to the IPs you provide. The script also runs docker run for vendor test images (nvidia/cuda, rocm/*) which will pull and execute container images from the network. These behaviors are coherent with the stated purpose but amount to network activity and code execution on the host (docker images) and attempts to access low‑level system interfaces that may require root. The SKILL.md does not explicitly warn about pulling/running container images or requiring root privileges for some checks.

Install Mechanism

No install spec — bundled as scripts. That keeps install risk low (no arbitrary archive download during install). Files are present in the skill package, so nothing is fetched during install; however runtime docker actions may fetch images from registries.

Credentials

No sensitive credentials are required. The primaryEnv is PREFLIGHT_NODE_ID which is just an identifier. The skill exposes many optional environment variables (PREFLIGHT_PEER_IPS, SWITCH_HOST, SWITCH_CLI_CMD, SWITCH_USER, etc.) which are relevant to cross‑node and switch checks. Nothing requests unrelated cloud/API credentials. However the registry's labeling of PREFLIGHT_NODE_ID as a "primary credential" is misleading.

Persistence & Privilege

always is false and the skill does not request permanent presence or modify other skills. It can be invoked autonomously (default) which is normal — note this combined with network and docker execution increases operational impact, but autonomy alone is not flagged.

What to consider before installing

This skill appears to do what it claims (local and cross‑node hardware/config checks), but take precautions before running it on production hosts: - Run it in a safe environment first (a non‑production node or an isolated VM) to observe behavior. - Be aware it may require root for full coverage (dmidecode, setpci, ipmitool, reading /dev/mem, etc.). Without root some checks will fail or be skipped. - Cross‑node checks (PREFLIGHT_PEER_IPS) will attempt SSH to the IPs you supply; those are outbound connections from the node. If you set SWITCH_HOST or similar it may attempt SSH to switches. - The script will run docker run with public vendor images (nvidia/cuda, rocm images). That will pull and execute container code from registries — review those image names and ensure pulling external images is acceptable in your environment. - The manifest under‑declares some runtime binaries (ssh, ip, lspci, ethtool, setpci, dmidecode, ipmitool, docker). Ensure required tooling is present and acceptable. - PREFLIGHT_NODE_ID is not a secret; the registry's labeling as a primary credential is misleading. If you want to proceed: review the bundled scripts (they are included), run with PREFLIGHT_PEER_IPS unset (local checks only) and without PREFLIGHT_STRICT first, and consider auditing or removing the docker tests if pulling/executing images is unacceptable in your environment.

By Xperf Inc. — part of the ClusterReady automatic cluster and rack validation platform.

AI-powered GPU cluster pre-flight validation skill. Run 26 GPU cluster node readiness checks locally on a bare-metal node. Auto-detects GPU vendor (NVIDIA / AMD) and network type (InfiniBand / RoCE / Ethernet). Validates GPU detection, PCIe topology, RDMA/InfiniBand networking, Docker, IOMMU, NUMA, firewall, BIOS, and more.

Example Prompts

• "Run a pre-flight check on this node"
• "Is my GPU node ready for training?"
• "Check cluster node health"
• "Validate this node before training"
• "Are my GPUs and network configured correctly?"
• "Run all health checks on this bare-metal node"
• "Check if GPUDirect and RDMA are working"
• "Is IOMMU in passthrough mode?"

When to Use

• Before running GPU workloads — ensure the node is properly configured
• After provisioning — validate new bare-metal nodes are cluster-ready
• Periodic health monitoring — catch hardware or config drift
• Troubleshooting — run specific checks to diagnose issues

How to Run

Run all checks:

bash {baseDir}/preflight.sh

Run specific checks only:

PREFLIGHT_CHECKS=1.3,1.4,1.5 bash {baseDir}/preflight.sh

Run in strict mode (no skippable failures):

PREFLIGHT_STRICT=true bash {baseDir}/preflight.sh

Enable cross-node checks:

PREFLIGHT_PEER_IPS=10.0.1.11,10.0.1.12 bash {baseDir}/preflight.sh

Environment Variables

Variable	Required	Default	Description
PREFLIGHT_NODE_ID	No	hostname	Node identifier in output
PREFLIGHT_CHECKS	No	all	Comma-separated check subset (e.g. 1.2,1.3,1.5)
PREFLIGHT_STRICT	No	false	Treat skippable failures as errors
PREFLIGHT_PEER_IPS	No	—	Comma-separated peer IPs for cross-node checks (1.1, 1.26, mesh ping)
MOUNT_POINT	No	auto-detect	Shared filesystem path for check 1.10
SWITCH_HOST	No	—	Switch hostname for check 1.25
SWITCH_CLI_CMD	No	—	Direct switch CLI command for check 1.25
SWITCH_USER	No	admin	Switch SSH user
SWITCH_SHOW_CMD	No	show interface status	Switch show command

Output

JSON to stdout, diagnostics to stderr. Pipe through jq for pretty-printing:

bash {baseDir}/preflight.sh 2>/dev/null | jq .

Exit Codes

• 0 — All checks passed (or skipped)
• 1 — One or more checks failed
• 2 — Configuration error

Check Catalog

ID	Name	Scope	Description
1.1	SSH Connectivity	Cross-node	Tests SSH to peer nodes
1.2	Docker Service	Local	Verifies Docker daemon is running
1.3	GPU Detection	Local	Detects NVIDIA or AMD GPUs
1.4	GPU Count and Model	Local	Reports GPU count and model
1.5	GPU Driver Version	Local	Reports GPU driver version
1.6	OS and Kernel	Local	Reports OS and kernel version
1.7	Network Type	Local	Detects InfiniBand, RoCE, or Ethernet
1.8	IB/NIC Ports	Local	Checks network port status
1.9	GPU Container Toolkit	Local	Tests GPU access inside Docker
1.10	Shared Filesystem	Local	Validates shared mount read/write
1.11	GPUDirect Kernel Module	Local	Checks RDMA peer memory modules
1.12	IOMMU Passthrough	Local	Validates IOMMU in passthrough mode
1.13	NUMA Balancing	Local	Reports NUMA balancing setting
1.14	Firewall Disabled	Local	Checks firewall is inactive
1.15	PCIe Link Speed/Width	Local	Validates PCIe link negotiation
1.16	PCIe ACS Disabled	Local	Checks ACS on PCI bridges
1.17	GPU-NIC PCIe Affinity	Local	Reports GPU/NIC topology
1.18	NIC Firmware	Local	Reports NIC firmware version
1.19	BIOS Settings	Local	Validates BIOS/IPMI settings
1.20	Link Quality	Local	Checks error counters
1.21	Transceiver Health	Local	Checks optical module health
1.22	MTU Configuration	Local	Validates jumbo frames (MTU >= 9000)
1.23	Fabric Topology	Local	Discovers IB fabric topology
1.24	RoCE QoS/NIC Config	Local	Reports QoS and GID configuration
1.25	Switch QoS	Cross-node	Validates switch configuration
1.26	Network Routing	Cross-node	Tests ping to peer nodes
mesh	L3 Mesh Ping	Cross-node	Tests connectivity to all peers

Interpreting Results

• pass — Check succeeded
• fail — Check failed (node may not be ready)
• skip — Check failed with a known non-critical signature (e.g. missing optional tool). In strict mode, these become failures.

Checks auto-detect GPU vendor (NVIDIA vs AMD) and network type (InfiniBand vs RoCE vs Ethernet), running the appropriate commands for each.

Support

• Website: xperf.ai
• Platform: clusterready.xperf.ai
• Contact: [e-mail]