Knowledge

The knowledge module provides a structured way to give agents access to factual information during phone calls. Organizations create knowledge bases, fill them with documents, and assign them to agents. During a call, the agent queries its knowledge bases via a dedicated LLM instead of guessing or hallucinating.

How It Works

Org creates Knowledge Base → Adds Documents → Assigns to Agent(s)

During a call:
  Agent receives a question it can't answer from its prompt alone
    → Calls query_knowledge tool with natural language query
    → All documents from linked KBs are sent to GPT-4o-mini
    → GPT-4o-mini extracts a concise answer grounded in the documents
    → Answer returned to the realtime voice model
    → Agent speaks the answer to the caller

This two-LLM architecture keeps the realtime model’s context window lean — documents aren’t stuffed into the voice session. Instead, a fast worker model (GPT-4o-mini) handles retrieval on demand.

Data Model

KnowledgeBase

pub struct KnowledgeBase {
    pub id: Uuid,
    pub title: String,
    pub document_count: usize,
    pub agent_ids: Vec<Uuid>,   // Agents this KB is assigned to
}

KnowledgeDocument

pub struct KnowledgeDocument {
    pub id: Uuid,
    pub knowledge_id: Uuid,     // Parent knowledge base
    pub title: String,
    pub description: Option<String>,
    pub content: String,        // Full document text
}

A knowledge base contains documents. Documents hold the actual content that gets queried. The agent_knowledge join table links agents to knowledge bases.

Query Pipeline

When an agent calls the query_knowledge tool during a live call:

1. Fetch documents — all documents from all knowledge bases linked to the agent
2. Format context — each document becomes ## Document: "{title}"\n\n{content}
3. Send to GPT-4o-mini with a system prompt enforcing grounded answers:
   • Answer concisely (will be read aloud during a phone call)
   • Use ONLY information from the provided documents
   • If no relevant info exists, say so
   • Don’t mention reading from documents — answer naturally
4. Return answer to the realtime voice model as the tool call result

Tool Registration

When an agent session starts, build_knowledge_tool_definition checks if the agent has linked knowledge bases. If so, it registers the query_knowledge tool and appends a behavioral prompt:

You have access to knowledge bases covering: {kb_titles}.
When the user asks about any of these topics, always use the query_knowledge
tool to look up accurate information — never guess or make up answers.

The tool description includes document titles and descriptions so the voice model knows what topics are available before deciding to call the tool.

Caution

All documents from linked knowledge bases are sent to GPT-4o-mini on every query. For agents with large document sets, this impacts latency and token usage.

API Endpoints

All endpoints require an authenticated session and scope to the org.

Knowledge Bases

Route	Method	Description
/api/knowledge	GET	List all KBs with document counts and agent assignments
/api/knowledge	POST	Create a KB (accepts title)
/api/knowledge/{id}	GET	Get a KB with its full document list
/api/knowledge/{id}	PUT	Update KB title
/api/knowledge/{id}	DELETE	Delete a KB
/api/knowledge/{id}/agents	PUT	Replace agent assignments (accepts Vec<Uuid>)

Documents

Route	Method	Description
/api/knowledge/{knowledge_id}/documents	POST	Add a document
/api/knowledge/{knowledge_id}/documents/{document_id}	PUT	Update title, description, content
/api/knowledge/{knowledge_id}/documents/{document_id}	DELETE	Delete a document

All document endpoints verify KB ownership before operating.

Module Structure

src/mods/knowledge/
├── api/        # CRUD for knowledge bases, documents, agent assignments
├── components/ # UI: KB list, card, details, document cards, create form
├── types/      # KnowledgeBase, KnowledgeDocument, data types
└── views/      # KnowledgeListView, KnowledgeDetailsView, CreateKnowledgeView

Related Modules

• Agent — agents link to KBs via agent_knowledge; the query_knowledge tool and prompt builder live in the agent module
• OpenAI — GPT-4o-mini powers the document query pipeline