A 900ms voice loop with a hard tool boundary

Sub-second is not one optimisation. It is a budget spent across turn detection, transcription, reasoning, tool execution and speech synthesis, and the naive implementation spends most of it in the wrong place.

Turn detection is what breaks first, and it breaks worst on exactly the calls that matter. A silence-threshold detector waits for a pause to decide the caller has finished. On a code-switched sentence — English into Hindi mid-clause — the natural pauses land in the wrong places, so the agent either interrupts the caller or sits waiting through a pause that was not an ending. Add background noise and the threshold stops meaning anything.

Interruption is a state problem, not an audio problem. When a caller talks over the agent, the system has to abandon its in-flight response, decide what the caller actually said, and continue the conversation without resetting mid-sentence. A loop that treats each turn as independent loses the thread here.

Context loading is on the critical path. An agent reasoning about "your policy" needs that policy in hand. Fetching it mid-turn spends latency budget at exactly the wrong moment.

And a free-running model on a recorded, regulated call is a liability in a way it is not in a chat box. Not because it is often wrong, but because when it is, it is wrong on a recording, inside a regulated process, and discovered at audit.

• Sub-second end to end, including transcription, reasoning, action and synthesis

• Turn detection that survives code-switching, noise and interruption

• No action without validation, permission and disclosure enforced before execution

• A structured record per call, not an audio file someone has to listen to

Three layers with hard boundaries: a real-time conversation loop, a typed tool layer, and an insurance data plane.

The conversation loop is tuned entirely for latency. Listen, detect the turn, transcribe, reason, act, reply. Turn detection is the component that received the most work, because it is the one that determines whether the other optimisations are perceptible at all — a 200ms saving in synthesis is invisible next to a detector that waits half a second too long to decide the caller stopped talking.

The tool layer is tuned entirely for correctness and is where every insurance operation lives: policy, claim, payment, endorsement. Each is a typed, schema-validated action carrying permission checks, required disclosures and escalation rules. The model's output is a request to call a tool; it is never itself an action.

The data plane models customers, policies, claims, premium transactions and voice sessions as structured entities, with context loaders that hydrate the agent with the applicable policy and claim data at call start — off the critical path — and persist decisions afterwards.

Voice workers and the API are separated into isolated containerised services, so the platform scales horizontally behind a load balancer as call volume rises without response time degrading under peak.

Design decision: compliance lives in the tool contract, not in the prompt. "Will the model always make the disclosure?" is unanswerable. "Can this operation execute without the disclosure?" is answerable, and the answer is no.

Load context at call start

Applicable policy and claim data is hydrated when the call opens, not fetched mid-turn. This takes the largest variable cost off the per-turn critical path.

Detect the turn

Tuned for call-centre conditions: code-switched speech, background noise, and callers who interrupt. The detector has to distinguish a mid-sentence pause from an ending, which is the difference between an agent that feels attentive and one that talks over people.

Transcribe and reason

Bilingual transcription feeds a reasoning step whose output is a tool request, not prose to be executed. The distinction matters at the boundary below.

Act through a typed tool

The requested operation is schema-validated, permission-checked, and gated on any required disclosure before it executes. A request that fails any of those does not execute — it escalates.

Reply, and persist

Speech synthesis returns the turn inside the budget, and the decision, the tool call and its result are written to the call record as they happen rather than reconstructed afterwards.

Governance

The voice model holds no direct authority over any insurance system. The typed tool set is the complete action surface; schema validation bounds what each action can contain; permission checks bound who it can be done for; disclosure gates bound when it may proceed at all; and escalation rules define where the agent must stop. Because the boundary is a contract rather than a behaviour, it can be reviewed by reading it.

Orchestration

Per turn: detect, transcribe, reason, validate, act, persist, synthesise — inside roughly 900ms. Across the call: context loaded once at open, state carried across turns so an interruption resumes rather than restarts, and escalation that hands a human the dialogue and the actions already taken.

Observability

Each call is a structured record rather than an audio artefact: turns, transcripts, tools invoked, arguments, results, disclosures made, escalations, and outcome. QA, claim verification and compliance review query the same record.

Latency is instrumented per stage, which matters because "the agent felt slow" is otherwise an unfalsifiable report. Per-stage timing tells you whether you are losing time in detection, transcription, reasoning, a slow tool, or synthesis — four different fixes.

The failures are conversational and integration failures, not reasoning failures.

Turn detection fires early and the agent interrupts a caller mid-thought. It fires late and the caller repeats themselves. Background noise pushes the detector's confidence down on exactly the calls where the customer is already frustrated. A code-switch mid-clause produces a transcript fragment that is grammatical in neither language.

On the action side, the designed behaviour is that a tool call failing validation, permission or a disclosure gate does not degrade into an approximation. It escalates. An agent that cannot complete a payment must hand off, never improvise something adjacent — that class of helpfulness is precisely what makes a voice agent unsafe in a regulated process.

Escalation carries state. A handoff that makes the customer repeat everything converts a partial success into a worse experience than not having answered at all.

Measured on the built system, in internal testing. This project did not reach deployment, so these are capability results rather than production service levels.

• End-to-end response from 2.5–3 seconds to around 900ms, on noisy bilingual calls with interruptions

• Turn stability through code-switching and interruption, without mid-sentence resets

• Every insurance action schema-validated, permissioned and logged before execution

• Horizontal scaling with response time held under peak, via separated voice-worker and API services

Directly demonstrable in the built system: the sub-second bilingual loop, turn detection under noise and interruption, the typed tool layer with its permission and disclosure gates, the insurance data plane and context loaders, and the per-call structured record.

Internal test result requiring reproduction: the ~900ms response time, down from 2.5–3 seconds. Latency is a property of the whole path — model, network, telephony, synthesis — so this number should be reproduced on your own path rather than assumed.

Status, stated plainly: a real client project that reached internal stage and was not deployed.

If you are building a voice agent for a regulated process, the component to budget for is turn detection, and the design question to settle first is where your compliance boundary lives. If the answer is "in the prompt", you do not have one.