Text-to-Speech (TTS)

Text to speech for voice agents is the stage that speaks the reply the model wrote. Text-to-speech, or TTS, turns written text into spoken audio. It is the reverse of speech-to-text, and in a voice agent built as a cascade, where separate models transcribe, reason and speak, it is the last stage of every turn: the language model writes the reply and TTS speaks it. Naturalness is largely solved. For a voice agent the open questions are how soon the first sound plays, whether a policy number or an amount comes out in a form the caller can check, and whether the audio actually says what the model wrote.

How neural TTS works#

Older systems stitched together fragments of recorded speech, which is why automated announcements had a seam between words. Neural TTS generates the audio instead. Tacotron 2, in 2018, paired a sequence-to-sequence model with a WaveNet vocoder and scored a mean opinion score (listeners rating naturalness from 1 to 5) of 4.53, against 4.58 for professionally recorded speech.

A current pipeline has three stages. Text is normalised and converted into phonemes or model tokens, so "Dr." becomes "doctor" and a date becomes words. An acoustic model predicts prosody, the rhythm, stress and intonation that carry meaning. A vocoder then renders the waveform, in small chunks so playback can start before the line is finished.

Watch the first stage most closely in a voice agent. Normalisation decides how "1234" is read, whether "St." is a street or a saint, and how a policy number with letters and digits is spoken. A model that sounds human and reads a reference number as "twelve thirty-four" has still failed the call.

Time to first audio#

For anything interactive, the metric is time to first audio: from sending text to the first sound the caller hears. As of October 2026 the vendor figures are model time only. ElevenLabs lists about 75 ms for its Flash model, excluding application and network latency. Cartesia says Sonic 3.6 streams the first byte of audio in about 90 ms. Deepgram gives under 200 ms to first byte for Aura-2. Measure end to end on your own network and telephony path before you compare them, because transit can exceed the model time.

Then check whether synthesis is where your time actually goes. On AVOX, the insurance voice agent we built, every stage of a turn was timed separately. Turn detection, the decision that the caller has finished, took most of the tuning, because a saving in synthesis is invisible next to a detector that waits too long. A team that picks a TTS vendor on a 75 ms headline before timing its own turns is optimising the wrong stage.

Overlap is the change that does matter in synthesis. Start speaking while the language model is still writing, by sending text to TTS at each sentence or clause boundary over a streaming connection. Waiting for the whole reply adds the full generation time to the silence, and the caller starts talking again just as the answer begins.

The API you stream through can change under you. OpenAI's TTS models, including tts-1 and gpt-4o-mini-tts, are scheduled to shut down on 6 January 2027, and the listed replacement, gpt-realtime-2.1-mini, runs on the Realtime API rather than the speech endpoint. OpenAI calls that a change in API workflow. Plan it as a rebuild of the synthesis connection, using the order of checks in how to replace the model under a production agent.

Speaking values a caller can check#

A caller hears a value once and cannot scroll back. Our voice agent guide reads back an irreversible action from the tool arguments, in a form the caller can check on the spot: "forty-two pounds fifty to the card ending 1234", not "your refund of the eligible amount". Give the day of the week with a date where it helps, and read reference numbers in short groups. Write those rules into the text you send to TTS, rather than hoping its normaliser guesses them.

In a cascade, text exists before audio, so code can check it. On AVOX the required disclosures sat in the tool contract, and a payment could not run until its disclosure had been given. A disclosure can be checked as a string before it is spoken, which a speech-to-speech model gives you no clean point to do.

Test the voice as well as the words. Re-transcribe the audio that was played and compare it with the text the agent meant to say, scoring names, numbers and dates on their own, as our voice agent testing guide describes. On a bilingual line, run that check in each language. On AVOX Realty, the real-estate voice agent we built on the same runtime, callers mixed Hindi and English inside a single clause.

Cloning, consent and disclosure#

Most TTS platforms can clone a voice from a short sample, and the same feature is a fraud tool. On 16 April 2026, Senator Maggie Hassan wrote to ElevenLabs, LOVO, Speechify and VEED asking what each does to stop scammers misusing its voice clones. ElevenLabs' safety page lists verification before its professional cloning tool, blocking of celebrity and other high-risk voices, and a classifier that detects audio made with ElevenLabs.

The rules are tightening. Article 50 of the EU AI Act, whose transparency duties began applying on 2 August 2026, requires that people be told when they are talking to an AI system and that generated audio be marked as such. Tennessee's ELVIS Act protects a person's voice likeness. OpenAI's usage requirement for its TTS is a clear disclosure to listeners that the voice is AI-generated.

If you clone a voice, get written consent from its owner and keep the recording of that consent. Tell callers the voice is synthetic at the start of the call. Prefer a provider that verifies identity before training a clone, because a provider without that check will also clone voices for people who have no consent to give.