Private Ai

Private AI — run LLMs, image generation, speech-to-text, and embeddings on your own hardware. Private AI keeps all data on your network. No cloud APIs, no telemetry, no third-party access. Offline AI and air-gapped AI compatible. On-premise AI for privacy, compliance, data sovereignty. HIPAA-friendly private AI, GDPR-ready private AI. 私有AI离线推理。IA privada sin nube.

Private AI — Your Data Never Leaves Your Network

Private AI means every prompt, every response, every image, every transcription stays on your machines. Private AI requires no cloud APIs. Private AI sends no telemetry. Private AI has no third-party access. This is offline AI that runs entirely on your own hardware.

What makes this private AI

• Private AI networking — the router and nodes communicate only on your local network
• Private AI telemetry — zero. No usage data, no analytics sent anywhere
• Private AI credentials — none needed. No API keys, no accounts, no tokens
• Offline AI capable — models run fully offline after download. Air-gapped AI deployment supported
• Private AI data storage — all data in ~/.fleet-manager/ on your machines only
• Air-gapped AI — pre-download models, then disconnect. The private AI fleet runs without internet

Setup private AI

bash

pip install ollama-herd    # install private AI router
herd                       # start private AI router (port 11435)
herd-node                  # run on each device — private AI nodes discover each other

No models downloaded during installation. All private AI model downloads require explicit user confirmation. Once downloaded, your private AI runs entirely offline.

Private AI for LLM inference

Send sensitive prompts with private AI — legal documents, medical notes, financial data, proprietary code never leave your network.

python

from openai import OpenAI

# Private AI endpoint — all inference stays local
private_ai = OpenAI(base_url="http://localhost:11435/v1", api_key="not-needed")

# Private AI document analysis — data never leaves your network
response = private_ai.chat.completions.create(
    model="llama3.3:70b",  # private AI inference on your hardware
    messages=[{"role": "user", "content": "Review this confidential contract for risks using private AI"}],
)
print(response.choices[0].message.content)

Private AI with any model — all fully offline

bash

# Private AI with DeepSeek-R1 — no data sent to cloud
curl http://localhost:11435/api/chat -d '{
  "model": "deepseek-r1:70b",
  "messages": [{"role": "user", "content": "Analyze this private financial report with offline AI"}],
  "stream": false
}'

Private AI image generation

Generate images from sensitive prompts with private AI — no DALL-E, no Midjourney, no cloud upload.

bash

# Private AI image generation — prompt stays local
curl http://localhost:11435/api/generate-image \
  -d '{"model": "z-image-turbo", "prompt": "confidential product mockup generated by private AI", "width": 1024, "height": 1024}'

Private AI transcription

Private AI transcription for meetings, legal depositions, medical dictation — no cloud STT.

bash

# Private AI speech-to-text — audio never leaves your network
curl http://localhost:11435/api/transcribe \
  -F "file=@private_board_meeting.wav" \
  -F "model=qwen3-asr"

Private AI embeddings

Build private AI knowledge bases from proprietary documents — no OpenAI embedding API.

bash

# Private AI embeddings — document content stays local
curl http://localhost:11435/api/embed \
  -d '{"model": "nomic-embed-text", "input": "Q4 revenue projections — private AI embedding for internal RAG"}'

Air-gapped AI / offline AI deployment

For fully disconnected private AI environments:

1. Download models on a connected machine: ollama pull llama3.3:70b
2. Transfer model files to the air-gapped network (USB, sneakernet)
3. Start the private AI fleet — runs without internet as offline AI

The private AI router discovers nodes on the local network. No DNS, no external lookups, no cloud callbacks. True offline AI.

Private AI compliance

Compliance Need	How private AI helps
HIPAA	Private AI processes PHI on your hardware only
GDPR	Private AI keeps all data in your jurisdiction
Data residency	Private AI — all processing on your hardware
No subprocessors	Private AI uses no cloud APIs for inference
Audit trail	Private AI logs every request (model, node, latency)
Data sovereignty	Private AI — your data, your machines, your control
Data minimization	Private AI traces store routing metadata, never prompts

Monitor your private AI fleet

bash

# Private AI fleet status
curl -s http://localhost:11435/fleet/status | python3 -m json.tool

# Private AI health checks
curl -s http://localhost:11435/dashboard/api/health | python3 -m json.tool

Private AI dashboard at http://localhost:11435/dashboard — accessible only on your local network.

Example private AI fleet response:

json

{
  "fleet": {"nodes_online": 3, "private_ai": true},
  "nodes": [
    {"node_id": "Private-AI-Server-1", "models_loaded": ["llama3.3:70b"]},
    {"node_id": "Private-AI-Server-2", "models_loaded": ["deepseek-r1:70b"]}
  ]
}

Full documentation

• Agent Setup Guide
• Configuration Reference
• API Reference

Contribute

Ollama Herd is open source (MIT). Private AI for everyone:

• Star on GitHub — help others discover private AI
• Open an issue — share your private AI setup
• PRs welcome — CLAUDE.md gives AI agents full context. 444 tests.

Guardrails

• No automatic downloads — all private AI model pulls require explicit user confirmation.
• Private AI model deletion requires explicit user confirmation.
• No external network access — private AI router and nodes communicate only locally.
• Private AI traces never store prompt content — only routing metadata.
• Never delete or modify files in ~/.fleet-manager/.