Local LLM Config

Sets up, manages, and optimizes locally-hosted LLMs (Ollama, LMStudio, llama.cpp) with detection, configuration, benchmarking, and model selection capabilities.

Local LLM Config Skill

When to use

Use when you need to configure, switch, tune, or benchmark locally-hosted LLMs (Ollama, LMStudio, llama.cpp, etc.) for any skill-enabled agent.

Essential for:

• Setting up development environments with local LLMs to avoid API costs
• Ensuring privacy by keeping all LLM processing on the local machine
• Optimizing model selection based on available hardware resources
• Switching between different models for various tasks (reasoning vs. speed)
• Troubleshooting LLM connectivity and performance issues
• Creating reproducible LLM configurations for team sharing

What it does

• 🔍 Detection: Automatically detects installed LLM backends via common CLI tools (ollama, lmstudio, llama-cli) or environment variables
• 📋 Listing: Lists available models with detailed specifications including size, quantization, and estimated memory requirements
• ⚙️ Configuration: Generates model-selection configuration files (YAML format) that agents can consume
• 📊 Benchmarking: Measures model performance including latency, token throughput, and resource utilization
• 🔧 Tuning: Provides helper snippets to generate prompt templates optimized for selected models
• 💾 Persistence: Stores configuration in the agent’s skill directory or profile for consumption by other skills
• 🔄 Management: Enables switching between models and backends without disrupting ongoing agent operations

Typical workflow

1. Detect: Run local-llm-config detect to see what LLM backends are available and what models they offer
2. List: Use local-llm-config list --verbose to see detailed model specifications for informed selection
3. Select: Choose and configure a model with local-llm-config select --model <model-name> [--backend <backend>]
4. Benchmark (optional): Verify performance with local-llm-config bench --model <model-name> [--timeout <seconds>]
5. Reference: Point your agent’s settings or downstream skills to the generated configuration file
6. Manage: Switch models later using the select command or view current config with show

Configuration Options

• $LOCAL_LLM_CONFIG_FILE: Path to custom configuration file (default: llm.yaml in skill directory)
• $LOCAL_LLM_CACHE_DIR: Directory for caching model information and benchmark results
• $LOCAL_LLM_VERBOSE: Enable verbose logging (equivalent to --verbose flag)

Supported Backends

• Ollama: ollama command-line interface
• LMStudio: Local model serving via LMStudio application
• llama.cpp: Direct execution via llama-cli or similar binaries
• Auto: Automatic detection of any available backend (default)

References

• https://ollama.com - Ollama LLM framework
• https://lmstudio.ai - LMStudio local LLM interface
• https://github.com/ggerganov/llama.cpp - llama.cpp efficient LLM inference
• Hugging Face model repositories for compatible model formats
• LiteLLM and LlamaIndex integration patterns for LLM configuration

Triggers

This skill activates when the user mentions:

• "configure local LLM"
• "set up ollama"
• "choose local model"
• "benchmark LLM performance"
• "switch llm backend"
• "detect available models"
• "local llm configuration"
• "manage llama.cpp"
• "lmstudio setup"
• "optimize model selection"