Ubuntu Ai

Ubuntu AI — build a local AI platform on Ubuntu and Debian. Ubuntu AI across x86 desktops, ARM edge devices, Raspberry Pi, Jetson Orin, and cloud VMs. Hetero...

Security Scan

The skill's instructions and requirements are internally consistent with building a local Ubuntu AI platform, but it asks users to run third-party install scripts and create system services which carry elevated risk and should be reviewed before running.

Purpose & Capability

Name/description (local Ubuntu AI, Ollama Herd router/node) matches the instructions: installing Ollama, pip installing ollama-herd, configuring systemd, opening a local port, and running local API calls. Required binaries (curl/wget, python3, pip, systemctl, apt, nvidia-smi) are appropriate for this purpose.

Instruction Scope

SKILL.md instructs actions expected for deploying a local inference stack: apt installs, pip installs, creating systemd units, enabling firewall rules, and calling local APIs. These actions require sudo and will persist on the host; the instructions do not attempt to read unrelated host files or request unrelated credentials, but they do open a network service (port 11435) which increases attack surface.

Install Mechanism

Although the skill has no install spec, the runtime instructions tell the user to run a remote installer via curl -fsSL https://ollama.ai/install.sh | sh and to pip install ollama-herd. Piping an external shell script to sh is high-risk because it executes third-party code fetched at runtime; the script and pip package should be inspected and obtained from trusted releases before execution.

Credentials

No environment variables or unrelated credentials are requested. The metadata's required binaries and config path hints (~/ .fleet-manager/latency.db, logs/herd.jsonl) are consistent with a fleet manager / local service and proportionate to the stated functionality.

Persistence & Privilege

Instructions create and enable systemd services (/etc/systemd/system/herd-*.service) and modify firewall rules, requiring sudo. This persistent, privileged installation is expected for a system service but increases risk if the installed binaries/scripts are malicious or compromised — inspect packages and service unit ExecStart targets before enabling.

Assessment

This skill appears to do what it says (set up an Ollama Herd router/node on Ubuntu), but it relies on executing a remote installer (curl | sh) and installing packages system-wide and as services. Before proceeding: inspect the installer script at https://ollama.ai/install.sh and the pip package source (ollama-herd) or install from reviewed release artifacts; consider performing the install in a VM or disposable machine first; review the created systemd unit ExecStart paths and file ownership; limit network exposure (don't open port 11435 to untrusted networks); and back up important data. If you cannot or do not want to audit the remote script, treat the install step as high-risk and avoid running it on sensitive systems.

Ubuntu AI — Build a Local AI Platform on Ubuntu

Build a complete local AI platform on Ubuntu. LLM inference, image generation, and embeddings across desktops, servers, Raspberry Pis, and Jetson boards. Ubuntu AI supports x86-64 and ARM in the same cluster — heterogeneous hardware, one endpoint. No cloud APIs, no subscriptions.

Why Ubuntu AI

• Largest Linux ecosystem — Ubuntu is the #1 Linux distribution for AI/ML workloads
• apt-get native — install dependencies with the Ubuntu package manager you know
• systemd integration — Ubuntu AI starts on boot, restarts on failure
• NVIDIA CUDA on Ubuntu — best GPU driver support on any Linux distribution
• ARM + x86 — Ubuntu AI runs on x86-64 servers and ARM devices (Raspberry Pi, Jetson)
• Free forever — Ubuntu is free, Ollama is free, Ollama Herd is free

Ubuntu AI quick start

# Ubuntu prerequisites
sudo apt update
sudo apt install python3-pip curl

# Install Ollama on Ubuntu
curl -fsSL https://ollama.ai/install.sh | sh

# Install Ubuntu AI router
pip install ollama-herd

# Start Ubuntu AI
herd          # Ubuntu AI router on port 11435
herd-node     # register this Ubuntu machine

On other Ubuntu/Debian machines:

pip install ollama-herd
herd-node     # auto-discovers the Ubuntu AI router

Ubuntu AI with NVIDIA CUDA

# Install NVIDIA drivers on Ubuntu
sudo apt install nvidia-driver-550
sudo reboot

# Verify Ubuntu NVIDIA CUDA
nvidia-smi

# Ollama on Ubuntu automatically detects NVIDIA CUDA
ollama ps    # should show GPU acceleration

Ubuntu AI systemd services

# Ubuntu AI router service
sudo tee /etc/systemd/system/herd-router.service << 'EOF'
[Unit]
Description=Ubuntu AI Router
After=network.target ollama.service

[Service]
Type=simple
ExecStart=/usr/local/bin/herd
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl enable --now herd-router

# Ubuntu AI node service
sudo tee /etc/systemd/system/herd-node.service << 'EOF'
[Unit]
Description=Ubuntu AI Node
After=network.target ollama.service

[Service]
Type=simple
ExecStart=/usr/local/bin/herd-node
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl enable --now herd-node

Use Ubuntu AI

OpenAI SDK

from openai import OpenAI

# Your Ubuntu AI endpoint
client = OpenAI(base_url="http://localhost:11435/v1", api_key="not-needed")

response = client.chat.completions.create(
    model="qwen3.5:32b",
    messages=[{"role": "user", "content": "Write an Ubuntu systemd timer for daily backups"}],
    stream=True,
)
for chunk in response:
    print(chunk.choices[0].delta.content or "", end="")

curl

# Ubuntu AI inference
curl http://localhost:11435/api/chat -d '{
  "model": "llama3.3:70b",
  "messages": [{"role": "user", "content": "Explain Ubuntu package management"}],
  "stream": false
}'

Ubuntu AI hardware guide

Ubuntu Device	GPU	Best Ubuntu AI models
Ubuntu desktop (RTX 4090)	24GB	llama3.3:70b, qwen3.5:32b
Ubuntu desktop (RTX 4080)	16GB	phi4, codestral, qwen3.5:14b
Ubuntu Server (A100)	80GB	deepseek-v3, qwen3.5:72b
Ubuntu Server (no GPU)	CPU	phi4-mini, gemma3:4b
Raspberry Pi 5 (Ubuntu)	CPU	gemma3:1b, phi4-mini — edge Ubuntu AI
Jetson Orin Nano (Ubuntu)	8GB shared	phi4, llama3.2:3b — ARM Ubuntu AI

Ubuntu AI supports x86-64 and ARM architectures. Heterogeneous Ubuntu AI clusters work automatically.

Ubuntu AI environment

# Optimize Ollama on Ubuntu
sudo systemctl edit ollama
# Add under [Service]:
#   Environment="OLLAMA_KEEP_ALIVE=-1"
#   Environment="OLLAMA_MAX_LOADED_MODELS=-1"
sudo systemctl restart ollama

Ubuntu AI firewall

# Ubuntu UFW
sudo ufw allow 11435/tcp
sudo ufw reload

Monitor Ubuntu AI

# Ubuntu AI fleet status
curl -s http://localhost:11435/fleet/status | python3 -m json.tool

# Ubuntu AI health — 15 automated checks
curl -s http://localhost:11435/dashboard/api/health | python3 -m json.tool

# Ubuntu AI logs
journalctl -u herd-router -f

Dashboard at http://localhost:11435/dashboard — live Ubuntu AI monitoring.

Also available on Ubuntu AI

Image generation

curl http://localhost:11435/api/generate-image \
  -d '{"model": "z-image-turbo", "prompt": "Ubuntu penguin mascot", "width": 1024, "height": 1024}'

Embeddings

curl http://localhost:11435/api/embed \
  -d '{"model": "nomic-embed-text", "input": "Ubuntu AI local inference Debian"}'

Full documentation

• Agent Setup Guide
• API Reference

Contribute

Ollama Herd is open source (MIT). Ubuntu AI users welcome:

• Star on GitHub
• Open an issue

Guardrails

• Ubuntu AI model downloads require explicit user confirmation.
• Ubuntu AI model deletion requires explicit user confirmation.
• Never delete or modify files in ~/.fleet-manager/.
• No models are downloaded automatically — all pulls are user-initiated or require opt-in.