Elevenlabs Twilio Memory Bridge

FastAPI personalization webhook that adds persistent caller memory and dynamic context injection to ElevenLabs Conversational AI agents on Twilio. No audio proxying, file-based persistence, OpenClaw compatible.

Security Scan

The code and runtime instructions mostly match the described webhook bridge, but there are metadata mismatches and an important operational security gap (unauthenticated endpoints / optional webhook verification) that could let an attacker inject prompt context — review and harden before installing.

Purpose & Capability

The code implements a FastAPI webhook that assembles a system prompt from a soul template, per-caller session state, memories, and notes — this aligns with the skill's stated purpose. However the registry metadata (top-level summary) claims 'Required env vars: none' and 'Primary credential: none', while SKILL.md and manifest.json declare ELEVENLABS_API_KEY, ELEVENLABS_AGENT_ID, OPENCLAW_API_BASE_URL, PUBLIC_BASE_URL (and optional WEBHOOK_SECRET). That metadata mismatch is inconsistent and should be corrected.

Instruction Scope

SKILL.md and the code instruct running a public webhook that persists JSON under ./data and exposes endpoints to add memories/notes. The code does not require authentication for the admin API endpoints (/api/memory/{phone_hash}, /api/notes) and the personalization webhook signature verification is optional (depends on WEBHOOK_SECRET). If the webhook secret is not set or endpoints are publicly reachable without further restrictions, an attacker can post arbitrary memory/notes which are injected into the system prompt (direct prompt injection of the agent). The instructions do recommend setting WEBHOOK_SECRET and TLS but do not mandate or show how to enforce authenticated access for the memory/notes endpoints.

Install Mechanism

No high-risk install mechanism is present: repository contains requirements.txt and typical pip install instructions. There are no downloads from untrusted URLs or extract steps. The skill is distributed as source files; install risk is standard for a Python web app.

Credentials

Requested environment variables (ELEVENLABS_API_KEY, ELEVENLABS_AGENT_ID, OPENCLAW_API_BASE_URL, PUBLIC_BASE_URL, optional WEBHOOK_SECRET, DATA_DIR, etc.) are appropriate for the described integration. The inconsistency is that the registry metadata claims none are required while manifest.json and SKILL.md list them — this is an incoherence in metadata, not the runtime behavior. The soul_template.md contains highly personal context (named individual 'Richard', family details) which is expected for personalization but is sensitive and should be reviewed before publishing or deploying.

Persistence & Privilege

The service writes persistent JSON files under DATA_DIR (./data by default) for sessions, memories, and notes. This is proportional to the feature (file-backed persistence). always:true is not set. The skill does not request elevated system privileges or attempt to modify other skills/configs. Ensure proper filesystem permissions and that the data directory is protected and backed up/erased per privacy requirements.

What to consider before installing

This implementation appears to do what it says, but take care before deploying: - Metadata mismatch: the registry listing claims no required env vars, but SKILL.md and manifest.json require ELEVENLABS_API_KEY, ELEVENLABS_AGENT_ID, OPENCLAW_API_BASE_URL, PUBLIC_BASE_URL (and optional WEBHOOK_SECRET). Treat the manifest/SKILL.md as authoritative and correct the registry metadata if you publish. - Mandatory webhook verification: set WEBHOOK_SECRET and configure the same secret in ElevenLabs so the service verifies HMAC signatures. Without this, anyone can POST to /webhook/personalize and mimic ElevenLabs. - Protect admin endpoints: /api/memory/{phone_hash} and /api/notes allow adding content that will be injected into the system prompt. These endpoints have no authentication in the provided code — run the service behind a firewall/reverse-proxy and require authentication (API key, IP allowlist, or at minimum basic auth) or remove these endpoints in public deployments. - Audit the soul_template: it contains sensitive personal details (names, health, family) — review and remove or sanitize any PII you don't want stored or exposed in a public repo or public endpoint. - Use TLS and network controls: run behind TLS (Cloudflare Tunnel, nginx with TLS), restrict access to known ElevenLabs IPs if possible, and put the service on a private network or auth-protected endpoint. - File permissions and backups: DATA_DIR holds persistent memories — ensure correct filesystem permissions, encryption at rest if needed, and retention/erasure policies for privacy compliance. - Least-privilege API keys: use scoped ElevenLabs/OpenClaw keys and rotate them regularly. If you harden the webhook (enforce HMAC, authenticate admin endpoints, and restrict network access) and correct the listing metadata, the skill is coherent with its stated purpose. Without those mitigations it presents a real risk of remote prompt injection and data manipulation.

Personalization webhook service for ElevenLabs + Twilio voice agents with persistent caller memory.

What It Does

When a call arrives on your Twilio number, ElevenLabs' native integration triggers this webhook. The bridge looks up the caller's history, loads long-term memory facts and daily context notes, combines them with a customizable soul/personality template, and returns everything as a system prompt override so your agent greets each caller with full context.

Architecture

• No audio proxying - ElevenLabs and Twilio handle media directly
• Webhook only - called once per inbound call to inject context
• File-based persistence - JSON files in ./data/, zero external dependencies
• OpenClaw compatible - works with any OpenAI-compatible LLM endpoint

Endpoints

Endpoint	Method	Purpose
/webhook/personalize	POST	ElevenLabs calls this on inbound call
/webhook/post-call	POST	Optional post-call cleanup
/api/memory/{phone_hash}	POST	Add long-term fact about a caller
/api/notes	POST	Add global or caller-scoped context note
/health	GET	Health check

Setup

1. Clone repo, pip install -r requirements.txt
2. Copy .env.example to .env, fill in secrets
3. Configure ElevenLabs agent with Custom LLM pointing to your OpenClaw instance
4. Enable system prompt + first message overrides in agent Security settings
5. Add webhook URL https://your-domain/webhook/personalize in ElevenLabs settings
6. Import Twilio number in ElevenLabs dashboard
7. Run: uvicorn app:app --host 0.0.0.0 --port 8000

Required Environment Variables

• ELEVENLABS_API_KEY - scoped ElevenLabs key
• ELEVENLABS_AGENT_ID - your agent ID
• OPENCLAW_API_BASE_URL - your OpenClaw instance URL
• PUBLIC_BASE_URL - publicly reachable URL of this service

Security

• All caller phone numbers are SHA-256 hashed before storage/logging
• Secrets loaded exclusively from environment variables
• Optional HMAC webhook signature verification
• Safe for public GitHub repos, no secrets in source