Private AI. Offline AI. 私有AI离线. IA Privada.

Private AI — run LLMs, image generation, speech-to-text, and embeddings on your own hardware. Private AI keeps all data on your network. No cloud APIs, no te...

Security Scan

The skill's instructions, required tools, and storage locations are consistent with a private, on-premise AI router; nothing requested is disproportionate to its stated purpose.

Purpose & Capability

The name/description match the instructions: it directs installing a local 'herd' router, running 'herd' and 'herd-node', and calling local endpoints on port 11435. Declared required bins (curl or wget, optional python3/pip) align with examples. Minor note: the docs reference the separate 'ollama' command for model downloads but 'ollama' is not listed in the metadata's required binaries; this is an operational convenience mismatch rather than evidence of malice.

Instruction Scope

Instructions stay within the stated purpose (install package, start local router, interact with localhost APIs, transfer model files for air-gapped setups). They reference local config paths (~/.fleet-manager/*) and show logging/audit behavior. Small inconsistencies: the doc claims "zero telemetry" yet documents local logging/audit trails, and it references use of the 'ollama' tool (not declared). The SKILL.md does not instruct reading unrelated system files or exfiltrating data.

Install Mechanism

This is an instruction-only skill (no install spec). The SKILL.md tells users to run 'pip install ollama-herd' which is a normal user-side install step; the registry itself does not perform downloads. Note for users: 'pip install' will fetch a package from PyPI (or configured index) — verify package provenance before running.

Credentials

No environment variables or external API keys are required. Examples use an OpenAI-compatible client pointed at localhost with a placeholder api_key, which is consistent with a drop-in local endpoint. The skill does not request unrelated credentials.

Persistence & Privilege

The skill is not set to always:true and does not request elevated platform privileges. It documents storing data under the user's home (~/.fleet-manager). Note: because runtime instructions interact with localhost endpoints, an agent invoking this skill will make local network calls — expected for this use case but worth awareness.

Assessment

This skill appears coherent with its stated aim of running an on-premise AI router. Before installing, verify the pip package name and publisher (to avoid typosquatting), confirm you trust the GitHub repository/docs linked, and inspect or back up ~/.fleet-manager after first run (it will contain logs and model metadata). Be aware the skill's runtime will call localhost endpoints (http://localhost:11435); that is intended, but any granted agent capability to invoke the skill will be able to interact with local services — only enable it if you trust the skill and the environment. Finally, note the docs reference the 'ollama' tool for model pulls; if you plan to follow that workflow, ensure you understand and trust that separate binary as well.

Private AI — Your Data Never Leaves Your Network

Private AI means every prompt, every response, every image, every transcription stays on your machines. Private AI requires no cloud APIs. Private AI sends no telemetry. Private AI has no third-party access. This is offline AI that runs entirely on your own hardware.

What makes this private AI

• Private AI networking — the router and nodes communicate only on your local network
• Private AI telemetry — zero. No usage data, no analytics sent anywhere
• Private AI credentials — none needed. No API keys, no accounts, no tokens
• Offline AI capable — models run fully offline after download. Air-gapped AI deployment supported
• Private AI data storage — all data in ~/.fleet-manager/ on your machines only
• Air-gapped AI — pre-download models, then disconnect. The private AI fleet runs without internet

Setup private AI

pip install ollama-herd    # install private AI router
herd                       # start private AI router (port 11435)
herd-node                  # run on each device — private AI nodes discover each other

No models downloaded during installation. All private AI model downloads require explicit user confirmation. Once downloaded, your private AI runs entirely offline.

Private AI for LLM inference

Send sensitive prompts with private AI — legal documents, medical notes, financial data, proprietary code never leave your network.

from openai import OpenAI

# Private AI endpoint — all inference stays local
private_ai = OpenAI(base_url="http://localhost:11435/v1", api_key="not-needed")

# Private AI document analysis — data never leaves your network
response = private_ai.chat.completions.create(
    model="llama3.3:70b",  # private AI inference on your hardware
    messages=[{"role": "user", "content": "Review this confidential contract for risks using private AI"}],
)
print(response.choices[0].message.content)

Private AI with any model — all fully offline

# Private AI with DeepSeek-R1 — no data sent to cloud
curl http://localhost:11435/api/chat -d '{
  "model": "deepseek-r1:70b",
  "messages": [{"role": "user", "content": "Analyze this private financial report with offline AI"}],
  "stream": false
}'

Private AI image generation

Generate images from sensitive prompts with private AI — no DALL-E, no Midjourney, no cloud upload.

# Private AI image generation — prompt stays local
curl http://localhost:11435/api/generate-image \
  -d '{"model": "z-image-turbo", "prompt": "confidential product mockup generated by private AI", "width": 1024, "height": 1024}'

Private AI transcription

Private AI transcription for meetings, legal depositions, medical dictation — no cloud STT.

# Private AI speech-to-text — audio never leaves your network
curl http://localhost:11435/api/transcribe \
  -F "file=@private_board_meeting.wav" \
  -F "model=qwen3-asr"

Private AI embeddings

Build private AI knowledge bases from proprietary documents — no OpenAI embedding API.

# Private AI embeddings — document content stays local
curl http://localhost:11435/api/embed \
  -d '{"model": "nomic-embed-text", "input": "Q4 revenue projections — private AI embedding for internal RAG"}'

Air-gapped AI / offline AI deployment

For fully disconnected private AI environments:

1. Download models on a connected machine: ollama pull llama3.3:70b
2. Transfer model files to the air-gapped network (USB, sneakernet)
3. Start the private AI fleet — runs without internet as offline AI

The private AI router discovers nodes on the local network. No DNS, no external lookups, no cloud callbacks. True offline AI.

Private AI compliance

Compliance Need	How private AI helps
HIPAA	Private AI processes PHI on your hardware only
GDPR	Private AI keeps all data in your jurisdiction
Data residency	Private AI — all processing on your hardware
No subprocessors	Private AI uses no cloud APIs for inference
Audit trail	Private AI logs every request (model, node, latency)
Data sovereignty	Private AI — your data, your machines, your control
Data minimization	Private AI traces store routing metadata, never prompts

Monitor your private AI fleet

# Private AI fleet status
curl -s http://localhost:11435/fleet/status | python3 -m json.tool

# Private AI health checks
curl -s http://localhost:11435/dashboard/api/health | python3 -m json.tool

Private AI dashboard at http://localhost:11435/dashboard — accessible only on your local network.

Example private AI fleet response:

{
  "fleet": {"nodes_online": 3, "private_ai": true},
  "nodes": [
    {"node_id": "Private-AI-Server-1", "models_loaded": ["llama3.3:70b"]},
    {"node_id": "Private-AI-Server-2", "models_loaded": ["deepseek-r1:70b"]}
  ]
}

Full documentation

• Agent Setup Guide
• Configuration Reference
• API Reference

Contribute

Ollama Herd is open source (MIT). Private AI for everyone:

• Star on GitHub — help others discover private AI
• Open an issue — share your private AI setup
• PRs welcome — CLAUDE.md gives AI agents full context. 412 tests.

Guardrails

• No automatic downloads — all private AI model pulls require explicit user confirmation.
• Private AI model deletion requires explicit user confirmation.
• No external network access — private AI router and nodes communicate only locally.
• Private AI traces never store prompt content — only routing metadata.
• Never delete or modify files in ~/.fleet-manager/.