Hallucination Guard – CertainLogic Verifier
Overview
CertainLogic Verifier is an open‑source, self‑hosted middleware layer that sits between your LLM calls and your application. It validates every AI response against a verified facts database, flags hallucinations, caches verified answers (bypassing the LLM), and provides cryptographic audit logs.
Key capabilities:
- 99%+ hallucination block rate – rule‑based checks + TF‑IDF memory search against your
facts_db
- 85‑98% token savings – semantic cache hits skip the LLM entirely
- Self‑hosted & air‑gapped – nothing leaves your infrastructure; ready for HIPAA/GDPR/SOC2/FedRAMP
- MIT licensed – no proprietary lock‑in; inspect every validation rule
- Deterministic grounding – same query → same verified answer, every time
- Cryptographic audit logs – SHA‑256 chained JSONL for compliance
Quick Start (2‑Minute Install)
# Clone the repository
git clone https://github.com/CertainLogicAI/hallucination-guard
cd hallucination-guard
# Set up Python environment
python3 -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txt
# Start the service
uvicorn main:app --host 0.0.0.0 --port 8000
Verify it's working:
curl -X POST http://localhost:8000/validate \
-d '{"query": "What is the price of GPT‑5?", "response": "\$200/month"}'
Installation Options
1. Docker (Recommended for Production)
docker build -t hallucination-guard .
docker run -p 8000:8000 hallucination-guard
2. Kubernetes/Helm
See deploy/helm/ in the repository for production‑ready Helm charts.
3. Systemd Service
A sample systemd unit file is included at deploy/systemd/hallucination-guard.service.
Configuration
Facts Database
The verifier checks responses against facts_db.json. Populate it with your domain‑specific verified facts.
Example entry:
{
"fact": "Python was created in 1991 by Guido van Rossum",
"category": "programming",
"source": "official Python history",
"verified_at": "2026‑04‑20"
}
Adding facts:
- Manually edit
facts_db.json
- Use the
/facts/add endpoint (POST with JSON)
- Bulk‑load from documents via the
/warming/extract endpoint
Environment Variables
Set these in .env or as environment variables:
PRODUCT_MODE=coder # coder|agent (determines rate limits)
OPENROUTER_API_KEY=your_key # Required for cache‑miss fallback
LOG_LEVEL=INFO # DEBUG|INFO|WARNING|ERROR
CACHE_DIR=./cache # Persistent cache storage
Usage
Validating a Single Response
import requests
response = requests.post(
"http://localhost:8000/validate",
json={
"query": "What year was Python created?",
"response": "Python was created in 1991."
}
)
print(response.json())
Integrating with AI Agent Pipelines
Place the verifier between your LLM call and your application logic:
def get_ai_response(query):
# 1. Check cache first
cache_check = requests.post("http://localhost:8000/cache/check",
json={"query": query})
if cache_check.json().get("cached"):
return cache_check.json()["response"]
# 2. Call LLM
llm_response = call_llm(query)
# 3. Validate
validation = requests.post("http://localhost:8000/validate",
json={"query": query, "response": llm_response})
if validation.json().get("valid"):
return llm_response
else:
# Handle hallucination
raise ValueError(f"Hallucination detected: {validation.json()}")
Cache Management
- View cache stats:
GET /cache/stats
- Clear cache:
POST /cache/clear
- Warm cache:
POST /warming/run (requires OpenRouter API key)
Advanced Features
Deterministic Memory Search
The verifier uses TF‑IDF similarity to match queries against known facts, even with paraphrasing.
Uncertainty Detection
Responses containing "I think", "might be", "not sure" are penalized and flagged for review.
Numeric‑Unit Matching
Checks that numeric values match known facts with correct units (e.g., "5 km" vs "5 miles").
Audit Logs
All validations are logged to audit_log.jsonl with SHA‑256 chaining for tamper evidence.
Resources
scripts/
install.sh – One‑line installer for Linux/macOSdocker-compose.yml – Multi‑service setup with PostgreSQL for audit logs
references/
api-reference.md – Complete API documentationfacts-schema.md – Facts database schema and validation rulesintegration-guide.md – Step‑by‑step integration with popular AI frameworks
assets/
sample-facts.json – Example facts database with 50+ verified entriesdocker-compose.prod.yml – Production‑ready Docker Compose configuration
Support & Community
License
MIT – see LICENSE in the repository.
