Distributed Inference

v1.0.4

Distributed inference for Llama, Qwen, DeepSeek across heterogeneous hardware. Self-hosted distributed inference — scatter requests across macOS, Linux, Wind...

1· 181·2 current·2 all-time
byTwin Geeks@twinsgeeks
MIT-0
Download zip
LicenseMIT-0 · Free to use, modify, and redistribute. No attribution required.
Security Scan
VirusTotalVirusTotal
stale
View report →
OpenClawOpenClaw
Benign
medium confidence
Purpose & Capability
The name/description (distributed inference for Ollama across local machines) matches the instructions: discover nodes via mDNS or explicit URL, route requests, score nodes, and record metrics. Declared metadata (curl/sqlite3, config paths for latency DB and logs) is coherent with this purpose. Minor inconsistency: the runtime instructions show installing via pip (pip install ollama-herd) and use python3 in examples, but pip/python3 are only listed as 'optionalBins' rather than required — in practice Python and pip will be needed to follow the provided install instructions.
Instruction Scope
SKILL.md instructs running a coordinator and node agents, collecting heartbeat data every 5s (CPU, memory, disk, loaded models, optional capacity scores), writing a latency sqlite DB and JSONL logs, and doing local network discovery via mDNS. Those actions are expected for this functionality but do involve collecting system metrics and using local network discovery (mDNS) — both are legitimate for a fleet manager but are sensitive operations the user should expect.
Install Mechanism
The skill is instruction-only (no install spec in the registry). The docs instruct users to 'pip install ollama-herd' (PyPI) and run herd/herd-node; installing an external pip package is a moderate-risk action if done automatically, but here the registry does not auto-install. Recommend reviewing the PyPI package and GitHub repo before installing.
Credentials
No credentials or secret env vars are requested. The only env variables referenced are feature/config flags (e.g., FLEET_NODE_ENABLE_CAPACITY_LEARNING, FLEET_CONTEXT_PROTECTION). The skill reads/writes local config paths (~/.fleet-manager/latency.db, ~/.fleet-manager/logs/herd.jsonl) which is proportional to the stated purpose but worth noting for privacy.
Persistence & Privilege
The skill is not force-enabled (always: false) and is user-invocable. It stores local state (DB and logs) and listens/sends on the local network, which is appropriate for this service. It does not request elevated platform privileges or modify other skills' configs.
Assessment
What to check before installing/running this skill: - Review the upstream package and source (https://github.com/geeks-accelerator/ollama-herd and the PyPI project) to confirm the code matches the documentation and contains no unexpected network calls or telemetry. - Expect the coordinator and node agents to collect and store system metrics and per-request traces in ~/.fleet-manager (latency.sqlite and JSONL logs); if that data is sensitive, review/rotate/secure those files. - The system uses mDNS (_fleet-manager._tcp.local.) for discovery and may broadcast/listen on the local network; if you are on an untrusted network, restrict/mask that behavior or use explicit --router-url instead. - The SKILL.md shows installing via pip and uses python3 in examples; ensure you have the intended Python/pip version and do not install packages as root without verifying the package. - Because there is no registry install spec, the skill itself won't auto-install code, but following its instructions will install a third‑party PyPI package — treat that as a separate action and audit it. - If you need higher assurance, request the exact PyPI version hash or a signed release and compare it to the repository before installing. These checks will reduce risk and confirm the skill is doing what its documentation claims.

Like a lobster shell, security has layers — review code before you run it.

apple-siliconvk97bxkt6cxg8wpgm1vz1jpz7xs840rz9coordinationvk976x7d6rm4zq8nreceq4hernx83e4eqdeepseekvk97bxkt6cxg8wpgm1vz1jpz7xs840rz9distributed-inferencevk97bxkt6cxg8wpgm1vz1jpz7xs840rz9fault-tolerancevk97bxkt6cxg8wpgm1vz1jpz7xs840rz9heterogeneousvk97av3gck0mdrv131ds6v80dwn83ws1glatestvk971b8pk29f912pm691w23j7kn8452ntllamavk97bxkt6cxg8wpgm1vz1jpz7xs840rz9llm-infrastructurevk976x7d6rm4zq8nreceq4hernx83e4eqlocal-aivk97bxkt6cxg8wpgm1vz1jpz7xs840rz9mac-minivk97bxkt6cxg8wpgm1vz1jpz7xs840rz9mac-studiovk97bxkt6cxg8wpgm1vz1jpz7xs840rz9multi-nodevk97bxkt6cxg8wpgm1vz1jpz7xs840rz9qwenvk97bxkt6cxg8wpgm1vz1jpz7xs840rz9schedulingvk97bxkt6cxg8wpgm1vz1jpz7xs840rz9scoringvk97avshj61feyge3915vacpgs583xkw8self-hostedvk97bxkt6cxg8wpgm1vz1jpz7xs840rz9

License

MIT-0
Free to use, modify, and redistribute. No attribution required.

Runtime requirements

globe Clawdis
OSmacOS · Linux · Windows
Any bincurl, sqlite3

Comments