Twilio

The twilio module handles all telephony through Twilio’s APIs. It receives incoming calls via webhooks, streams audio bidirectionally over WebSocket, processes recordings, and manages phone numbers.

Call Flow

Incoming call → Twilio
  → POST /twilio/voice
    → TwiML: start recording (mono) + open WebSocket stream
  → WSS /twilio/stream
    → start event → resolve agent, create call record, open AI session
    → media events → forward audio to AI provider
    → AI audio delta → send back to Twilio stream
    → speech started → clear Twilio audio buffer (interruption)
  → Call ends → stop event → close sessions
  → POST /twilio/recording (async)
    → download recording → transcribe → analyze → report → todos → contacts

Webhooks & Routes

Path	Method	Handler	Auth
/twilio/voice	POST	twilio_voice	None (Twilio webhook)
/twilio/stream	WSS	twilio_stream	None (Twilio WebSocket)
/twilio/recording	POST	twilio_recording	None (Twilio webhook)
/api/twilio/buy-phone-number	POST	buy_phone_number_api	Session required

Voice Webhook

Receives the initial call from Twilio. Returns TwiML that:

1. Starts a mono recording with a status callback to /twilio/recording
2. Opens a WebSocket media stream to /twilio/stream with from and to as custom parameters

WebSocket Stream

The core real-time handler. On upgrade:

1. Wait for start event — reads messages until a valid Start arrives (discards Connected and others)
2. Resolve agent — calls handle_twilio_stream_in_event_start:
   • Looks up phone number → finds assigned agent
   • Loads agent’s realtime_config to determine provider
   • Resolves or creates a contact for the caller
   • Creates a call record in the database
3. Create AI session — opens WebSocket to OpenAI or Gemini based on config
4. Run two concurrent loops (tokio::select!):
   • Twilio listener — forwards audio from caller to AI via send_audio_delta
   • Realtime listener — dispatches AI events:
     • AudioDelta → send audio to Twilio (skipped if interrupted)
     • SpeechStarted → set interrupted flag, send clear to flush Twilio buffer
     • ResponseStarted → reset interrupted flag
     • ToolCall → execute tool, send result back to AI

Recording Webhook

Fires asynchronously after the call ends. Spawns a background task (tokio::spawn) that:

1. Downloads the MP3 recording using the org’s Twilio credentials
2. Saves the audio file ({call_sid}.mp3)
3. Transcribes via transcribe_audio (GPT-4o Transcribe)
4. Saves the transcript ({call_sid}.txt)
5. Updates the call record with recording_url and transcription
6. Runs post-call pipeline (all fire-and-forget, errors logged but not propagated):
   • enrich_contact — update contact with call context
   • analyze_call — run all org analyzers
   • extract_todos — AI-powered to-do extraction
   • send_call_report — email report

Stream Events

Inbound (Twilio → Loquent)

Event	Type	Key Fields
connected	TwilioStreamInEventConnected	protocol, version
start	TwilioStreamInEventStart	stream_sid, call_sid, from, to, media_format
media	TwilioStreamInEventMedia	payload (base64 µ-law), sequence_number
stop	TwilioStreamInEventStop	call_sid, account_sid

Outbound (Loquent → Twilio)

Event	Type	Purpose
media	TwilioStreamOutEventMedia	Audio chunk for playback
clear	TwilioStreamOutEventClear	Flush buffered audio (interruption)

Audio format: base64-encoded µ-law, 8kHz, mono.

Phone Number Management

Utility	What It Does
buy_twilio_phone_number	POST to Twilio API to purchase a number by area code
create_twilio_subaccount	Create a per-org Twilio subaccount
set_phone_number_twiml	Set the voice webhook URL on a phone number

The buy flow also assigns default analyzers to the new number and configures the webhook URL.

Configuration

Platform-Level (core_conf)

pub struct TwilioCoreConf {
    pub twilio_main_sid: String,
    pub twilio_main_token: String,
}

Per-Organization (organization table)

Field	Purpose
twilio_account_sid	Org’s Twilio subaccount SID
twilio_auth_token	Org’s Twilio auth token

Constants

Constant	Value
TWILIO_API_BASE	https://api.twilio.com/2010-04-01
TWILIO_VOICE_PATH	/twilio/voice
TWILIO_STREAM_PATH	/twilio/stream
TWILIO_RECORDING_PATH	/twilio/recording

Module Structure

src/mods/twilio/
├── api/        # Voice webhook, recording webhook, buy number
├── conf/       # TwilioCoreConf (from core_conf table)
├── constants/  # API base URL, webhook paths
├── routes/     # WebSocket stream handler (the core loop)
├── services/   # Start event handling, media forwarding
├── types/      # Stream events (in/out), call data, phone numbers, recording
└── utils/      # Buy number, create subaccount, process recording, set TwiML

Related Modules

• Agent — resolved from phone number, provides realtime config and tools
• OpenAI — receives audio stream for AI processing
• Call — call records created during stream start
• Contact — caller resolution and enrichment
• Analyzer — post-call analysis triggered after recording