Phone Voice Integration

Connect ElevenLabs Agents to your OpenClaw via phone with Twilio. Includes caller ID auth, voice PIN security, call screening, memory injection, and cost tracking.

Security Scan

The instructions broadly match a phone-to-agent bridge but the skill under-declares required secrets, local file access, and exposes your machine to the network — review carefully before installing or running.

Purpose & Capability

The stated purpose (connect ElevenLabs + Twilio + Anthropic via a bridge) reasonably requires Twilio, ElevenLabs, and Anthropic credentials and a network tunnel. However the registry metadata claims no required environment variables or config paths while the SKILL.md clearly expects many secrets and local files (MEMORY.md, USER.md, .env, contacts.json). That mismatch is notable.

Instruction Scope

SKILL.md instructs the agent/operator to load and inject local sensitive files (MEMORY.md, USER.md, recent transcripts) into prompts, to store voice PINs and other secrets in .env, and to transmit conversation data to external services (ElevenLabs, Anthropic, Twilio). It also suggests optional live data (calendar, weather) via external CLIs/APIs. These actions read and transmit sensitive local data not declared in the skill metadata and could leak private information if misconfigured.

Install Mechanism

This is instruction-only (no install spec or code), which reduces direct install risk. The doc recommends installing cloudflared or using ngrok (standard tooling) via brew/ngrok; that is expected for exposing a local bridge but increases exposure. Because there's no provided code to review, the bridge implementation is unspecified — you must supply or review that code yourself.

Credentials

Although the registry lists no required env vars, the instructions require multiple high-privilege secrets (ElevenLabs API key/xi-api-key, Twilio Account SID/Auth Token, Anthropic API key, a bridge auth token, possibly calendar API tokens). Requesting all of these is reasonable for the feature set, but the skill under-declares them and gives no guidance on least-privilege, token scoping, or secure storage. Storing PINs and tokens in plaintext .env files is explicitly suggested, which is risky.

Persistence & Privilege

The skill does not request 'always' privilege, which is good, but it instructs you to run a permanent Cloudflare tunnel or ngrok exposing a local server to the internet. That materially increases attack surface and persistence of external access to your machine and any files the bridge can read. The bridge will handle credentials and transcripts, so run it in an isolated environment and ensure proper access controls.

What to consider before installing

This skill's purpose is plausible, but the SKILL.md expects you to provide and expose several sensitive credentials and local data while running a public-facing bridge — and the registry metadata does not declare those requirements. Before installing or running anything: 1) Do not copy unreviewed bridge code from unknown sources — implement or review the FastAPI bridge yourself. 2) Use dedicated, least-privilege API keys (separate accounts or scoped tokens) for Twilio/ElevenLabs/Anthropic and rotate them after testing. 3) Avoid storing long-term secrets in plaintext .env; use a secrets manager if possible and prefer short-lived tokens. 4) Run the bridge in an isolated VM/container with strict network/firewall rules and logging. 5) Be cautious about memory files (MEMORY.md, USER.md, transcripts) — they contain personal data and will be sent to external LLM services; redact or limit sensitive content. 6) Prefer temporary tunnels (ngrok short-lived) or carefully configured Cloudflare tunnels, and restrict incoming origins and auth. 7) Ask the publisher for the bridge source code, an explicit list of required env vars, and instructions for secure deployment; if they cannot provide it, consider this skill suspicious and avoid running it in production. Additional info that would raise confidence to 'high': the actual bridge source code for review, a declared env var list in registry metadata, and documented token scoping/retention policies.

Turn your OpenClaw into a phone-callable assistant with ElevenLabs Agents + Twilio.

What you get:

• 📞 Call your bot from any phone
• 🔐 Caller ID authentication + voice PIN security
• 🛡️ Call screening (whitelist-based)
• 🧠 Full memory context (loads MEMORY.md, USER.md)
• 💰 Cost tracking per call
• 📝 Call transcripts with summaries
• ⏱️ Rate limiting
• 🌐 Permanent tunnel (Cloudflare) or temporary (ngrok)

Architecture

Phone → Twilio → ElevenLabs Agent → Your Bridge → Anthropic Claude → OpenClaw Tools
                                          ↓
                                    Memory Context
                                    (MEMORY.md, USER.md)

Flow:

1. Caller dials your Twilio number
2. Twilio routes to ElevenLabs Agent
3. Agent sends chat completions to your bridge (mimics OpenAI API)
4. Bridge translates to Anthropic, injects context from memory files
5. Claude response → ElevenLabs TTS → caller hears it

Prerequisites

• OpenClaw installed and running
• ElevenLabs account + API key
• Twilio account + phone number
• Anthropic API key
• Cloudflare tunnel or ngrok (for exposing localhost)

Setup

1. Enable Chat Completions in OpenClaw

Not needed for this skill — the bridge bypasses OpenClaw and calls Claude directly. This gives you more control over memory injection and cost tracking.

2. Create the Bridge Server

The bridge is a FastAPI server that:

• Accepts OpenAI-compatible /v1/chat/completions requests from ElevenLabs
• Injects memory context (MEMORY.md, USER.md, live data)
• Calls Anthropic Claude API
• Streams responses back in OpenAI format
• Logs costs and transcripts

Key files:

• server.py — FastAPI app with /v1/chat/completions endpoint
• fred_prompt.py — System prompt builder (loads memory files)
• .env — Secrets (API keys, tokens, whitelist)
• contacts.json — Caller whitelist for screening

3. Set Up Cloudflare Tunnel (Recommended)

Permanent, secure alternative to ngrok:

# Install cloudflared
brew install cloudflare/cloudflare/cloudflared

# Login and configure
cloudflared tunnel login
cloudflared tunnel create <tunnel-name>

# Run the tunnel
cloudflared tunnel --url http://localhost:8013 run <tunnel-name>

Add a CNAME in Cloudflare DNS:

voice.yourdomain.com → <tunnel-id>.cfargotunnel.com

Or use ngrok (temporary):

ngrok http 8013

4. Configure ElevenLabs Agent

Option A: Manual (UI)

1. Go to ElevenLabs dashboard → Conversational AI
2. Create new agent
3. Under LLM settings → Custom LLM
4. Set URL: https://voice.yourdomain.com/v1/chat/completions
5. Add header: Authorization: Bearer <YOUR_BRIDGE_TOKEN>

Option B: Programmatic (API)

# Step 1: Store your bridge auth token as a secret
curl -X POST https://api.elevenlabs.io/v1/convai/secrets \
  -H "xi-api-key: YOUR_ELEVENLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "new",
    "name": "bridge_auth_token",
    "value": "YOUR_BRIDGE_AUTH_TOKEN"
  }'

# Response: {"secret_id": "abc123..."}

# Step 2: Create the agent
curl -X POST https://api.elevenlabs.io/v1/convai/agents/create \
  -H "xi-api-key: YOUR_ELEVENLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "conversation_config": {
      "agent": {
        "language": "en",
        "prompt": {
          "llm": "custom-llm",
          "prompt": "You are a helpful voice assistant.",
          "custom_llm": {
            "url": "https://voice.yourdomain.com/v1/chat/completions",
            "api_key": {"secret_id": "abc123..."}
          }
        }
      }
    }
  }'

5. Connect Twilio Phone Number

In ElevenLabs agent settings:

1. Go to Phone section
2. Enter Twilio Account SID and Auth Token
3. Select your Twilio phone number
4. Save

Done! Your bot now answers that phone number.

Security Features

Caller ID Authentication

Recognizes whitelisted numbers automatically:

// contacts.json
{
  "[phone]": {
    "name": "Alice",
    "role": "family"
  }
}

Voice PIN Challenge

For unknown callers or high-security actions:

VOICE_PIN = "banana"  # Set in .env

Caller must say the PIN to proceed.

Call Screening

Unknown numbers get a receptionist prompt:

"This is Fred's assistant. I can take a message or help with general questions."

Rate Limiting

Configurable per-hour limits:

RATE_LIMIT_PER_HOUR = 10

Prevents abuse and runaway costs.

Memory Injection

The bridge auto-loads context before each call:

Files read:

• MEMORY.md — Long-term facts about user, projects, preferences
• USER.md — User profile (name, location, timezone)
• Recent call transcripts (cross-call memory)

Live data injection:

• Current time/date
• Weather (optional, via API)
• Calendar events (optional, via gog CLI)

All injected into the system prompt before Claude sees the conversation.

Cost Tracking

Every call logs to memory/voice-calls/costs.jsonl:

{
  "call_sid": "CA123...",
  "timestamp": "2026-02-03T10:30:00",
  "caller": "[phone]",
  "duration_sec": 45,
  "total_cost_usd": 0.12,
  "breakdown": {
    "twilio": 0.02,
    "elevenlabs": 0.08,
    "anthropic": 0.02
  }
}

Run analytics on the JSONL to track monthly spend.

Usage Example

Call your bot:

1. Dial your Twilio number
2. If you're whitelisted → casual conversation starts
3. If you're unknown → receptionist mode
4. Ask it to check your calendar, send a message, set a reminder, etc.

Outbound calling (optional):

curl -X POST https://voice.yourdomain.com/call/outbound \
  -H "Authorization: Bearer <BRIDGE_TOKEN>" \
  -d '{"to": "[phone]", "message": "Reminder: dentist at 3pm"}'

Configuration Options

Environment variables (.env):

ANTHROPIC_API_KEY=sk-ant-...
ELEVENLABS_API_KEY=sk_...
ELEVENLABS_AGENT_ID=agent_...
TWILIO_ACCOUNT_SID=AC...
TWILIO_AUTH_TOKEN=...
TWILIO_NUMBER=+1...
LLM_BRIDGE_TOKEN=<random-secure-token>
VOICE_PIN=<your-secret-word>
CLAWD_DIR=/path/to/clawd

Whitelist (contacts.json):

{
  "[phone]": {"name": "Alice", "role": "family"},
  "[phone]": {"name": "Bob", "role": "friend"}
}

Advanced: Office Hours

Restrict calls to business hours:

# In server.py
OFFICE_HOURS = {
    "enabled": True,
    "timezone": "America/Vancouver",
    "weekdays": {"start": "09:00", "end": "17:00"},
    "weekends": False
}

Outside hours → voicemail prompt.

Debugging

Test the bridge directly:

curl -X POST https://voice.yourdomain.com/v1/chat/completions \
  -H "Authorization: Bearer <BRIDGE_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-4",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": false
  }'

Check logs:

tail -f ~/clawd/memory/voice-calls/bridge.log

Verify Twilio webhook:

1. Call your number
2. Check Twilio console → Call logs → Webhook status
3. Should see 200 responses from ElevenLabs

Cost Estimates

Per-minute breakdown:

• Twilio: ~$0.01/min (inbound) + carrier fees
• ElevenLabs TTS: ~$0.05/min (varies by voice quality)
• Anthropic Claude: ~$0.01/min (depends on token usage)
• Total: ~$0.07-0.10/min (~$4-6/hour of talk time)

Use rate limiting and call screening to control costs.

Comparison: This vs Basic Tutorial

ElevenLabs official tutorial:

• ✅ Basic integration
• ❌ No security
• ❌ No memory persistence
• ❌ No cost tracking
• ❌ Temporary ngrok URL

This skill (Phone Voice v2.0):

• ✅ All of the above
• ✅ Caller ID + PIN security
• ✅ Cross-call memory
• ✅ Cost tracking & analytics
• ✅ Permanent tunnel (Cloudflare)
• ✅ Rate limiting
• ✅ Call screening
• ✅ Transcript logging

Links

• ElevenLabs Agents: https://elevenlabs.io/conversational-ai
• Twilio: https://www.twilio.com/
• Cloudflare Tunnels: https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/
• Reference implementation: (Available on request — DM @FredMolty)

License

MIT — use freely, credit appreciated.

---

Built by Fred (@FredMolty) — running on OpenClaw since 2026.