Agent

The agent module defines AI voice agents that handle incoming phone calls. Each agent has an identity, a system prompt, a realtime provider configuration, and optional knowledge bases.

Data Model

Agent

pub struct Agent {
    pub id: Uuid,
    pub name: String,
    pub prompt: String,
    pub realtime_config: AgentRealtimeConfig,
    pub knowledge_base_ids: Vec<Uuid>,
}

AgentData (Create/Update Payload)

pub struct AgentData {
    pub name: String,
    pub prompt: String,
    pub realtime_config: AgentRealtimeConfig,
    pub knowledge_base_ids: Vec<Uuid>,
}

API Endpoints

Method	Path	Description
GET	/api/agents	List all agents for the organization
GET	/api/agents/:id	Get agent with knowledge base IDs
POST	/api/agents	Create agent + link knowledge bases
PUT	/api/agents/:id	Update agent (replaces all knowledge base links)
DELETE	/api/agents/:id	Delete agent

All endpoints require an authenticated Session. Agents are scoped to the session’s organization.

Note

GET /api/agents returns agents with empty knowledge_base_ids. Use GET /api/agents/:id for the full agent with linked knowledge bases.

Realtime Configuration

AgentRealtimeConfig is a tagged enum — the provider field determines which AI backend handles voice sessions.

OpenAI Provider

{
  "provider": "openai",
  "model": "gpt-realtime",
  "voice": "alloy",
  "speed": 1.0,
  "noise_reduction": "near_field",
  "vad": {
    "type": "semantic_vad",
    "eagerness": "medium",
    "create_response": true,
    "interrupt_response": true
  }
}

Setting	Options	Default
model	gpt-realtime	gpt-realtime
voice	alloy, ash, ballad, coral, echo, sage, shimmer, verse	alloy
speed	f32	1.0
noise_reduction	near_field, far_field, none	near_field
vad.type	semantic_vad, server_vad	semantic_vad
vad.eagerness	low, medium, high, auto	medium

Server VAD exposes additional fields: threshold, prefix_padding_ms, silence_duration_ms, idle_timeout_ms.

Gemini Provider

{
  "provider": "gemini",
  "model": "gemini-live-2.5-flash-preview-native-audio-09-2025",
  "voice": "Aoede",
  "start_of_speech_sensitivity": "START_SENSITIVITY_HIGH",
  "end_of_speech_sensitivity": "END_SENSITIVITY_HIGH",
  "activity_handling": "START_OF_ACTIVITY_INTERRUPTS"
}

Setting	Options	Default
voice	Aoede, Charon, Fenrir, Kore, Puck	Aoede
start_of_speech_sensitivity	START_SENSITIVITY_HIGH, START_SENSITIVITY_LOW	HIGH
end_of_speech_sensitivity	END_SENSITIVITY_HIGH, END_SENSITIVITY_LOW	HIGH
silence_duration_ms	Option<u32>	None
activity_handling	START_OF_ACTIVITY_INTERRUPTS, NO_INTERRUPTION	INTERRUPTS

Agent Tools

During a live call, agents can execute two types of tools:

query_knowledge

Queries the agent’s linked knowledge bases using LLM-powered search. Built dynamically by build_knowledge_tool_definition — only registered if the agent has linked knowledge bases.

The tool collects all documents from linked knowledge bases, sends them with the query to GPT-4o Mini, and returns a concise voice-friendly answer.

lookup_caller

Looks up caller information via the organization’s external client lookup URL. Only registered if organization.client_lookup_url is configured.

Sends a GET request with ?phone=<from_number> and an optional loquent_x_check auth header. Returns the raw response body to the AI model.

Realtime Session Traits

The module defines provider-agnostic traits for realtime communication:

#[async_trait]
pub trait RealtimeSessionSender: Send + Sync {
    async fn send_audio_delta(&mut self, audio_delta: String) -> Result<...>;
    async fn send_tool_response(&mut self, call_id: String, function_name: String, output: String) -> Result<...>;
}

#[async_trait]
pub trait RealtimeSessionReceiver: Send {
    async fn receive_event(&mut self) -> Result<RealtimeInEvent, ...>;
}

Events received from any provider are normalized into RealtimeInEvent:

Variant	Meaning
AudioDelta	Audio chunk to relay to the caller
SpeechStarted	User interrupted — model cancelled its response
ResponseStarted	Model started a new response
ToolCall	Model requesting a function call
Unknown	Any other event (ignored)

Module Structure

src/mods/agent/
├── api/          # CRUD endpoints (create, get, list, update, delete)
├── components/   # UI components (list, details, card, realtime config form)
├── services/     # Session creation, tool collection, tool handling, event processing
├── traits/       # RealtimeSessionSender, RealtimeSessionReceiver
├── types/        # Agent, AgentRealtimeConfig, tool definitions, events
└── views/        # Page views (list, details, create)

Related Modules

• Twilio — delivers incoming calls to agents
• OpenAI — implements sender/receiver for OpenAI Realtime
• Gemini — implements sender/receiver for Gemini Live
• Knowledge — knowledge bases queried by query_knowledge tool