Text-to-Speech (TTS)

Text-to-speech, or TTS, is the mirror image of speech-to-text: it takes written text and produces spoken audio. It narrates audiobooks, reads notifications aloud, and gives a realtime voice agent its voice. Two things changed in the last few years. Neural models made synthetic voices genuinely hard to distinguish from recordings, and streaming synthesis made them fast enough to hold a live conversation. Both matter, but they matter to different products: an audiobook pipeline cares about expressiveness per dollar, a phone agent cares about how many milliseconds pass before the first audio plays.

From robotic to indistinguishable#

Older systems were concatenative: they stitched together fragments of recorded speech, which is why automated announcements had that telltale seam between words. Neural TTS replaced stitching with generation. DeepMind's WaveNet showed in 2016 that a network could model the raw waveform directly, and Tacotron 2 in 2018 paired a sequence-to-sequence model with a WaveNet vocoder and scored a mean opinion score of 4.53 against 4.58 for professionally recorded human speech. The gap has only narrowed since.

A modern pipeline has three conceptual stages. Text is first normalized and converted into phonemes or model tokens (so "Dr." becomes "doctor" and "2026" becomes words). An acoustic model then predicts prosody, the rhythm, stress, and intonation that carry meaning, as a sequence of intermediate representations, today usually discrete tokens from a neural audio codec. Finally a codec decoder or vocoder renders those tokens into a waveform. Because prosody is learned from data rather than hand-written rules, the model gets questions, emphasis, and pacing right, and the newest generation infers emotional delivery from the text itself: ElevenLabs v3 even accepts inline audio tags like a whisper or a laugh.

Streaming and time-to-first-audio#

For anything interactive you cannot wait for a full sentence to render before playback starts. Streaming synthesis emits audio in small chunks as soon as the first words are ready, and the metric that decides whether an agent feels responsive is time-to-first-audio (TTFA): how long from sending text to hearing sound. As of mid-2026 the fast tier looks like this: Cartesia's Sonic models advertise roughly 40 ms of model latency (they use a state space architecture rather than a transformer, which holds up well under load), ElevenLabs Flash sits around 75 ms of model inference, Deepgram Aura-2 lands under 200 ms and can be tuned to about 90, and general cloud TTS typically measures 150 to 300 ms. Add network transit and real end-to-end TTFA runs higher than any vendor headline.

Two implementation details matter as much as the model. First, transport: ElevenLabs and Cartesia offer WebSocket streaming where you push text fragments and pull audio continuously, while OpenAI's TTS endpoint streams over plain HTTP responses, fine for one-shot synthesis but clumsier for incremental input. Second, overlap: in a voice agent you should start synthesizing while the upstream LLM is still generating, flushing text to TTS at sentence or clause boundaries. With a 75 ms TTS model behind a streaming LLM, speech can begin playing about 100 to 150 ms after the first sentence of the reply completes, instead of seconds after the whole answer does. See latency metrics for how this composes with time-to-first-token.

Voice cloning, consent, and misuse#

Most serious TTS platforms can clone a voice. Instant cloning works from seconds of audio and gets you a recognizable timbre; professional cloning trains on thirty minutes to a few hours of clean recordings and gets you a voice the speaker's own family would accept. Quality tracks input quality: studio audio yields a faithful clone, phone-quality clips yield a muddy one.

The same capability is a fraud tool, which is why this section is not optional reading. Voice phishing using cloned voices of executives and relatives is now a documented attack pattern, and in April 2026 US senators formally pressed ElevenLabs and its peers on consent verification, watermarking, and provenance. The reputable providers have converged on a similar defense stack: identity verification before professional cloning, blocklists for celebrity and political voices (ElevenLabs calls these no-go voices), and classifiers that can trace generated audio back to the account that made it. Regulation is catching up too: the EU AI Act requires disclosure of synthetic media, and US state laws like Tennessee's ELVIS Act protect voice likeness directly.

If you build with cloning: get written consent from the voice owner, keep the recording of that consent, disclose to listeners that the voice is synthetic (OpenAI makes this disclosure a usage requirement), and prefer providers that verify identity rather than ones that will clone anyone from a YouTube clip.

The provider landscape, mid-2026#

ElevenLabs remains the default for expressiveness, with the largest voice library, 70+ languages, and Flash for the low-latency tier, at premium prices. Cartesia competes on raw speed and consistency under load, around $38 per million characters. OpenAI's gpt-4o-mini-tts and the realtime models are the simplest choice if you already run on OpenAI, with steerable accent, emotion, and pacing through instructions. Deepgram Aura-2 targets enterprise telephony, trained on call-center audio and deployable on-prem at about $30 per million characters. Hume specializes in emotionally expressive delivery, and open-weight models keep improving for teams that need self-hosting and zero per-character fees at the cost of running their own GPUs. Pick along four axes: TTFA, expressiveness, deployment constraints, and price per character, because no provider currently wins all four.

Practical takeaways#

Treat TTS as solved for naturalness and unsolved for everything around it. Benchmark time-to-first-audio end to end on your own network, not from vendor pages. Stream at every boundary: WebSocket transport, sentence-level flushing from the LLM, chunked playback. Handle voice cloning as a consent obligation with a paper trail, not just a feature flag. And re-run your provider comparison every couple of quarters, because the latency and price leaderboard in this market has flipped more than once since 2024.