Contact Memory

Contact memory gives agents persistent context across interactions. Every memory-writing source — post-call summaries, message analysis, enrichment, and plan executor observations — routes through a single AI-powered pipeline that merges, deduplicates, and formats the result.

Data Model

AgentContactNote

pub struct AgentContactNote {
    pub id: String,
    pub content: String,
    pub is_system: bool,     // true = evolving system memory
    pub created_at: String,
    pub updated_at: String,
}

Agents receive all notes (interaction + system) as AgentContactNote — a simplified DTO without UI concerns like author_name or can_edit. Notes are ordered chronologically.

The Unified Memory Pipeline

All AI-powered memory updates flow through update_contact_memory() — the single entry point that ensures consistent formatting, merging, and deduplication regardless of the source.

New information arrives (call summary / messages / enrichment / plan executor)
  → update_contact_memory(contact_id, org_id, &content, source)
      → Load existing system note (if any)
      → Build contact profile context
      → LLM synthesizes merged memory following MEMORY_FORMAT_INSTRUCTIONS
      → Upsert single system note (is_system: true, area: Memory)
      → Log AI usage (feature: update_contact_memory, area per source)

Memory Update Sources

The MemoryUpdateSource enum identifies the origin of new information and selects the appropriate AI model area, usage tracking, and prompt framing:

pub enum MemoryUpdateSource {
    /// After-call transcription summary.
    Call,
    /// Batch-analyzed message conversation transcript.
    Messages,
    /// Enrichment data extracted from conversation history.
    Enrichment,
    /// Plan executor's update_system_note tool call.
    UpdateSystemNote,
}

Source	AI Area	Usage Feature	Source Label
`Call`	`UpdateSystemNote`	`UpdateContactMemory`	”New interaction”
`Messages`	`UpdateContactMemoryFromMessages`	`UpdateContactMemoryFromMessages`	”New messages”
`Enrichment`	`UpdateSystemNote`	`UpdateContactMemory`	”New enrichment data”
`UpdateSystemNote`	`UpdateSystemNote`	`UpdateContactMemory`	”New agent observations”

Post-Call Pipeline

After every call with a transcription, the memory pipeline runs automatically:

Call ends → Transcription saved
  → summarize_call_for_note(transcription)
      LLM produces a structured markdown summary:
        ## Summary, ## Key Points, ## Next Steps

  → write_interaction_note(contact_id, summary)
      Creates a user-visible note (is_system: false)
      Author label: "AI Agent"

  → update_contact_memory(contact_id, org_id, &summary, MemoryUpdateSource::Call)
      Merges new facts into the system note via AI synthesis

Plan Executor Route

The plan executor’s update_system_note tool routes through the same pipeline. When the executor calls the tool during plan execution:

LLM calls update_system_note({ contact_id, content: "Prefers morning calls" })
  → Guard: contact must belong to the plan
  → update_contact_memory(contact_id, org_id, &content, MemoryUpdateSource::UpdateSystemNote)
  → On success: log ToolCallVariantOutput::Done to plan timeline
  → On failure: log ToolCallVariantOutput::Failed + tracing::warn!

The tool description instructs the LLM to pass only new facts — the memory system handles merging with existing memory. This replaces the previous approach where the executor called upsert_contact_memory() directly with a full summary, bypassing AI synthesis.

System Note Format

The system note follows a structured section-based markdown format. All sources produce the same structure via MEMORY_FORMAT_INSTRUCTIONS:

**Name:** Sarah Chen
**Language:** English

## Relationship
- Customer since January 2026, referred by Jane Doe

## Communication style
- Prefers brief, direct messages
- Quick responder on SMS, slow on email

## Preferences
- Morning calls only (before 11 AM EST)
- Interested in enterprise tier pricing

## Situation
- Evaluating Loquent against two competitors
- Data residency is a hard requirement (EU region)

## Pending
- Follow up with written quote by March 1

## Key dates
- Contract renewal: May 2026

Sections are optional — the LLM omits any section with no relevant information. The total is capped at 500 words.

Rules enforced by the prompt:

One fact per bullet point, third person
Never duplicate structured contact fields (email, phone, company, job title, tags, custom fields)
Drop stale or redundant information on each update

Real-Time Context Injection

Before the agent sends its first greeting, all contact notes are injected as conversation context via the provider API:

// In twilio_stream_route.rs, after session creation
get_contact_notes_for_agent(contact_id)
  // Returns all notes (interaction + system), chronologically
  → Format context block:
      "## Caller Context\n\n[system] <timestamp>: ...\n[interaction] <timestamp>: ..."
  → send_conversation_item(formatted_context)
  → trigger_response()  // Agent greets with full context

Agents are instructed (DO NOT EXPLICITLY MENTION THE NOTES OR QUOTE THEM) to use memory as background awareness rather than reciting it directly.

Visibility: System Notes in UI

System notes are visible in the contact detail view as read-only entries pinned to the top.

Note type	`is_system`	Label in UI	Editable	Visible to agent
User note	`false`	Author’s name	✅ By author	✅
Interaction note (AI)	`false`	AI Agent	❌	✅
System memory	`true`	Contact Memory	❌	✅

Key Files

File	Purpose
`src/mods/contact/services/notes/update_contact_memory_service.rs`	Unified memory pipeline, `MemoryUpdateSource` enum
`src/mods/plan/tools/build_tools.rs`	Plan executor’s `update_system_note` tool handler
`src/mods/contact/services/notes/`	`upsert_contact_memory` (direct writes without AI synthesis)

Contact Notes — user-visible notes, markdown rendering, and labels
Contact — parent module overview
AI Voice Agents — how agents use injected context during live calls
Plan Module — executor tools including update_system_note
Update Contact Tool — executor tool for profile field updates