SMS Sync

Loquent handles SMS in two ways: real-time delivery via an event sink (for new messages going forward) and a manual history sync (to import past messages from Twilio).

Real-Time SMS: Event Sink Setup

The event sink connects Twilio’s Event Streams to Loquent, delivering inbound message receipts and outbound delivery status updates in real time.

What it does to your Twilio account

Setting up the event sink does modify your Twilio account. It creates:

1. A Twilio Event Sink pointing to https://{APP_HOST}/twilio/events
2. A Twilio Event Subscription for these event types:
   • com.twilio.messaging.inbound-message.received
   • com.twilio.messaging.message.sent
   • com.twilio.messaging.message.delivered
   • com.twilio.messaging.message.failed
   • com.twilio.messaging.message.undelivered

The operation is idempotent — if a sink pointing to the same URL already exists, Loquent reuses it instead of creating a duplicate.

This does not affect any existing webhook configurations on your individual phone numbers.

How to set it up

1. Go to Settings → Twilio
2. Find the SMS Event Forwarding section
3. Click Set Up Event Sink

POST /api/twilio/setup-event-sink

How inbound messages are processed

When Twilio fires inbound-message.received to /twilio/events:

1. Loquent looks up the phone number from the to field
2. Resolves or creates a contact from the from number
3. Creates a message record: direction=inbound, status=received
4. Stores the Twilio Message SID in message.external_id
5. Triggers the text agent handler for AI response

How delivery status updates work

For sent, delivered, failed, and undelivered events:

1. Loquent looks up the existing message by external_id (Twilio Message SID)
2. Updates message.status to the new value

For messages sent through Loquent, the same message row tracks the full delivery lifecycle. For messages sent via other platforms, see External SMS Capture.

Message History Sync

For importing past SMS conversations from your Twilio account into Loquent.

What it does to your Twilio account

Nothing. The history sync is entirely read-only — it queries Twilio’s Messages API and writes the results to Loquent’s database. No changes are made in Twilio.

How to sync

1. Go to Settings → Twilio
2. Find the Message History section
3. Each imported phone number has its own Sync History button, or use Sync All Numbers to bulk sync
4. Optionally set a “since” date to limit the range

POST /api/twilio/sync-message-history
Input: phone_number_id (UUID), since (optional date)

How the sync works

For each direction (inbound + outbound):
  GET /Accounts/{sid}/Messages.json?To={number}&PageSize=100
  GET /Accounts/{sid}/Messages.json?From={number}&PageSize=100

  → Follows next_page_uri for pagination
  → Filters by DateSent>= if "since" provided
  → Deduplicates by Twilio Message SID

Batch 1: SELECT existing messages by external_id (IN clause)
  → Identifies already-imported SIDs

Batch 2: SELECT contacts by phone number (IN clause)
  → Creates stub contacts for unknown numbers (source="sms_sync")

Batch 3: INSERT messages in chunks of 500
  → ON CONFLICT DO NOTHING on (organization_id, external_id)
  → Preserves original Twilio timestamps via created_at override

Returns: { imported: N, skipped: N }

Running the sync multiple times is safe — already-imported messages are skipped by SID.

After inserting new messages, the sync updates last_contacted_at on affected contacts. Each contact’s timestamp is set to their most recent message — but only if the synced timestamp is newer than the existing value.

Note

There are no background or scheduled sync jobs. All message history sync is user-triggered from the Settings UI.

MMS Media Downloads

When synced messages contain MMS attachments (num_media > 0), Loquent downloads the media files after message insertion:

1. Filters imported messages with num_media > 0
2. Downloads each attachment from Twilio’s Media API with bounded concurrency (5 at a time)
3. Stores files in object storage and creates message_attachment records
4. Tracks progress via DownloadingMedia WebSocket events

Failed downloads log a warning but don’t block the sync. The response includes a media_downloaded count.

Real-Time Progress Events

The sync publishes progress over WebSocket so the UI can show live status updates. Events are scoped to the organization’s real-time channel.

Event type: twilio.sms_sync.progress

Phases:

Phase	Description
Fetching	Querying Twilio Messages API
Deduplicating	Checking for already-imported SIDs
ResolvingContacts	Creating or matching contacts
Inserting	Batch inserting new messages
UpdatingTimestamps	Updating contact last_contacted_at
DownloadingMedia	Downloading MMS attachments (includes completed/total counters)
Done	Sync complete

Payload:

pub struct SmsSyncProgress {
    pub phone_number_id: Uuid,
    pub phase: SmsSyncPhase,
    pub fetched: Option<usize>,
    pub new_count: Option<usize>,
    pub media_downloaded: Option<usize>,
}

The frontend subscribes via RealtimeContext and updates the UI per phone number. State transitions to “Done” only when the Done event arrives — not when the HTTP response returns — so the UI correctly waits for background media downloads to finish.

Parallel Sync All

Sync All Numbers fires all per-number sync requests concurrently using join_all(). Each number shows independent real-time progress. Results are aggregated into a combined total after all numbers complete.

Message Data Model

CREATE TABLE message (
    id               UUID PRIMARY KEY,
    organization_id  UUID NOT NULL,
    contact_id       UUID,                     -- Resolved contact (nullable)
    phone_number_id  UUID,                     -- Loquent phone number involved
    channel          TEXT,                     -- "sms"
    direction        TEXT,                     -- "inbound" | "outbound"
    body             TEXT,                     -- Message text
    from_address     TEXT,                     -- Sender phone number
    to_address       TEXT,                     -- Recipient phone number
    status           TEXT,                     -- "received" | "sent" | "delivered" | "failed" | "undelivered"
    external_id      TEXT,                     -- Twilio Message SID (indexed, used for status updates)
    created_at       TIMESTAMP                -- Original send time (overridden for synced messages)
);

-- Unique index for idempotent batch inserts (NULLs are distinct)
CREATE UNIQUE INDEX idx_message_org_external_id_unique
    ON message (organization_id, external_id);

API Reference

Method	Path	Description
POST	/api/twilio/setup-event-sink	Create Event Sink + Subscription in Twilio
POST	/api/twilio/sync-message-history	Import past messages from Twilio
GET	/api/twilio/imported-numbers	List imported phone numbers for the org
POST	/api/messages/send	Send an outbound SMS
GET	/api/messages	List messages for a contact

Webhook endpoints (Twilio → Loquent)

Path	Purpose
POST /twilio/events	Receives SMS event sink events

Module Structure

src/mods/twilio/
├── api/
│   ├── get_imported_phone_numbers_api.rs         # GET /api/twilio/imported-numbers
│   ├── setup_twilio_event_sink_api.rs
│   ├── sync_message_history_api.rs               # Sync + MMS download + progress events
│   └── twilio_events_api.rs                     # /twilio/events webhook
├── types/
│   ├── sync_message_history_progress_type.rs    # SmsSyncPhase + SmsSyncProgress
│   └── sync_message_history_response_type.rs    # Response with media_downloaded count
└── utils/
    ├── setup_org_event_sink_util.rs
    ├── create_event_sink_util.rs
    ├── create_event_subscription_util.rs
    ├── list_event_sinks_util.rs                 # Idempotency check
    └── fetch_twilio_message_history_util.rs     # Paginated message fetch

src/mods/contact/services/
└── update_contacts_last_contacted_batch_service.rs  # Batch timestamp updates

src/mods/messaging/
└── api/
    └── send_message_api.rs

src/mods/settings/components/
└── sync_message_history_section_component.rs    # Realtime progress UI