Voice agent latency measurement framework
The useful voice agent latency number is the time from the caller finishing a turn to the first audible agent response. Trace it across turn detection, streaming speech-to-text, LLM first token, streaming text-to-speech and media transport. This page defines the clocks and test controls; it does not combine measurements from different benchmark runs into a synthetic latency budget.
Which clocks should a latency trace record?
| Stage | Clock | Production check | Evidence |
|---|---|---|---|
| Turn detection | Last caller audio to a committed end-of-turn decision | Test pauses and backchannels, not only clean read speech. | Turn-taking boardSmart Turn method |
| Streaming STT | Committed turn to the final transcript used by the agent | Record partial and final timestamps. A fast partial is not a usable final. | Streaming STT board |
| LLM | Final transcript available to the first usable output token | Measure first token on the prompts, tools and guardrails used in calls. | LLM boardModel size test |
| Streaming TTS | First speakable text chunk to the first playable audio | Measure first audio and long-form stability separately. | TTS boardSpeed and quality profile |
| Full response | Last caller audio to the first audible agent response | Trace the complete call in one region; do not add medians from separate runs. | Speech-to-speech boardConversation latency test |
How should production latency be measured?
- Timestamp the last caller audio, committed transcript, first LLM token, first TTS byte and first played audio.
- Run the same prompts, tools, voices, regions and network path for every candidate.
- Publish sample size, run date, median and tail latency. Keep errors and empty responses in the denominator.
- Replay pauses, backchannels, noisy audio and tool calls from the calls the agent must actually handle.
What usually moves the number?
- Endpointing can add delay before any model request begins.
- LLM tokens per second do not answer the first-token question.
- TTS must receive a speakable chunk before it can stream audio.
- Region placement and media playback add time outside the model APIs.
- A model or provider change should be evaluated on the full call, not a single component score.
Cascade or speech-to-speech?
A cascade exposes separate STT, LLM and TTS clocks and lets the application replace one component. A speech-to-speech model can shorten the path, but it changes the observability, cost and component-control tradeoff. Compare both on the same complete conversation before changing the architecture.
Further evidence
Read the measured cascade and speech-to-speech comparison, then inspect the voice agent infrastructure map.