Lowest Latency Phone

Low-latency inbound and outbound AI phone calls via the OpenAI Realtime API and Twilio, covering pre-warm and pre-accept patterns, IVR and receptionist flows...

Security Scan

The skill's requirements and instructions are coherent with its stated purpose (OpenAI Realtime + Twilio phone integration); nothing requests unrelated credentials or installs arbitrary code, though the skill embeds system-level guidance the agent will follow so review before enabling.

Purpose & Capability

Name/description match the declared env requirements (OPENAI_API_KEY and Twilio credentials) and the runtime instructions. All required secrets are expected for a realtime Twilio+OpenAI phone integration; no unrelated services or superfluous binaries are requested.

Instruction Scope

SKILL.md is comprehensive and instructs the agent how to configure flows, open websockets, manage pre-warm/pre-accept, and how to call tools (DB, CRM, Twilio). This is expected for an instruction-only skill, but the SKILL.md contains system-style directives (e.g., 'How to answer questions' and explicit system prompts for modes) which function like a system prompt and will influence agent behavior—review these prompts and any dynamic tool hooks you implement. The instructions reference only files bundled in the package and environment variables that are declared or optional in the docs.

Install Mechanism

Instruction-only package with no install spec and no external downloads; lowest-risk install surface (nothing is written to disk beyond the skill files you intentionally copy).

Credentials

Required environment variables are limited and proportionate: OPENAI_API_KEY (primary) and Twilio account SID/auth token/phone number for the Twilio integration. Additional optional envs (DIDs, routing numbers, voice/model defaults) are documented as optional. No unrelated credentials or filesystem paths are requested.

Persistence & Privilege

Skill is user-invocable, not always-enabled, and allows autonomous model invocation (platform default). It does not request permanent 'always: true' or any special system-wide modifications. It does show how to set envs in openclaw.json (normal config behavior).

Scan Findings in Context

[system-prompt-override] expected: The SKILL.md contains system-style instructions (explicit guidance on how the agent should answer and mode system prompts). This is expected for a skill that supplies detailed behavior for an agent, but such content effectively acts as a system prompt and will shape responses—review carefully before trusting.

Assessment

This package is internally consistent with its stated purpose, but because it embeds system-style prompts and detailed runtime guidance you should: 1) review SKILL.md and the mode-specific system prompts to ensure they reflect acceptable behavior for your deployment; 2) only provide the minimum-scoped OpenAI and Twilio credentials needed (do not reuse high-privilege keys); 3) test in an isolated environment before enabling for production (especially CSR tools that access customer DBs); 4) treat prompt caching and any prompt content potentially containing PII/PHI with care; and 5) if you do not trust autonomous invocation, keep the skill disabled or restrict when it may run. The pre-scan flagged 'system-prompt-override' is expected here but is a reminder to verify prompts and tool implementations you connect to the skill.

Lowest Latency Calls

This skill teaches the agent how to configure, deploy, debug, and optimize an OpenAI Realtime API voice agent with Twilio for inbound and outbound AI phone calls, with a strong emphasis on reducing time-to-first-audio.

Research note: this package combines vendor documentation with thoroughly sourced practitioner research current as of 2026-04-02. Treat operational latency and behavior claims as field guidance to validate in your own stack, not as vendor guarantees.

When to invoke this skill

Invoke when the user asks about any of the following:

• Configuring or deploying a Realtime API voice agent (any name/brand)
• Outbound call pre-warm architecture (eliminating dead air at pickup)
• Inbound call pre-accept warm (eliminating dead air when answering)
• AI IVR: routing, menu trees, warm-transfer, conference-bridge handoff
• Claw Receptionist mode: greet, qualify, route or take messages
• CSR with DB mode: customer lookup, appointment booking, CRM notes
• Async tool calling — AI continues speaking while tools run in background
• VAD tuning — semantic vs server VAD, eagerness, mid-session switching
• Prompt caching (prompt_cache_key) for Realtime sessions
• gpt-realtime-1.5 vs gpt-realtime-mini vs legacy realtime model trade-offs
• Twilio Media Streams: PCMU format, edge colocation, AMD
• Known bugs, regressions, or latency issues
• Deploying or configuring this skill

Reference documents

All reference docs live in {baseDir}/docs/:

File	Content
{baseDir}/docs/01-overview.md	Model landscape, gpt-realtime-1.5 changelog, regressions
{baseDir}/docs/02-session-config.md	Full session.update reference + recommended defaults
{baseDir}/docs/03-prewarm-outbound.md	Pre-warm for outbound: buffer, fallback, edge cases
{baseDir}/docs/04-inbound-modes.md	Inbound: AI IVR, Claw Receptionist, CSR with DB
{baseDir}/docs/05-async-tools.md	Async tool calling for both directions
{baseDir}/docs/06-latency-tuning.md	All latency levers after pre-warm/pre-accept
{baseDir}/docs/07-twilio-integration.md	PCMU format, edge colocation, AMD, stream events
{baseDir}/docs/08-known-issues.md	Bugs, regressions, workarounds, watch-later items
{baseDir}/docs/09-openclaw-config.md	openclaw.json config + install/publish instructions

How to answer questions

1. Load the relevant doc from {baseDir}/docs/ before answering.
2. Always specify: direction (inbound/outbound), mode (IVR/Receptionist/CSR/raw), and which model version the behaviour applies to.
3. For latency questions, distinguish: initialization latency (pre-warm/pre-accept), turn latency (VAD + inference), and Twilio transport latency.
4. For configuration questions, provide exact JSON/JS code blocks.
5. Flag caveats from 08-known-issues.md whenever recommending a feature.
6. For async tool use, always reference 05-async-tools.md.
7. For inbound mode selection, reference 04-inbound-modes.md.

Key facts (always available without file load)

• OpenAI currently lists gpt-realtime-1.5 as its flagship voice model and gpt-realtime-mini as a lower-cost realtime option.
• Example WebSocket endpoint: wss://api.openai.com/v1/realtime?model=gpt-realtime-1.5
• Twilio Media Streams use mu-law / PCMU at 8 kHz mono, and the OpenAI-side examples in this package use audio/pcmu.
• Pre-warm (outbound) and pre-accept warm (inbound) are the core latency techniques in this package.
• semantic_vad with eagerness: "high" is presented here as a tested starting point, not a universal best setting.
• A 10-second pre-warm / pre-accept timeout is a conservative fallback pattern.
• Inbound modes covered: AI IVR, receptionist, and CSR / database-assisted flows.