Measurement
Voice agent latency measurement framework
A production framework for tracing voice agent latency across turn detection, streaming STT, LLM first token, TTS first audio and transport.
The useful voice agent latency number is the time from the caller finishing a turn to the first audible agent response. Trace it across turn detection, streaming speech-to-text, LLM first token, streaming text-to-speech and media transport. This page defines the clocks and test controls; it does not combine measurements from different benchmark runs into a synthetic latency budget.
- Measurement rule
- Keep one end-to-end clock. Component clocks explain the result; they do not replace it.
- Author
- Speko engineering
- Reviewed by
- Speko engineering
- Published
- Last reviewed
Which clocks should a latency trace record?
| Stage | Clock | Production check | Evidence |
|---|---|---|---|
| Turn detection | ClockLast caller audio to a committed end-of-turn decision | Production checkTest pauses and backchannels, not only clean read speech. | EvidenceTurn-taking boardSmart Turn method |
| Streaming STT | ClockCommitted turn to the final transcript used by the agent | Production checkRecord partial and final timestamps. A fast partial is not a usable final. | EvidenceStreaming STT board |
| LLM | ClockFinal transcript available to the first usable output token | Production checkMeasure first token on the prompts, tools and guardrails used in calls. | EvidenceLLM boardModel size test |
| Streaming TTS | ClockFirst speakable text chunk to the first playable audio | Production checkMeasure first audio and long-form stability separately. | EvidenceTTS boardSpeed and quality profile |
| Full response | ClockLast caller audio to the first audible agent response | Production checkTrace the complete call in one region; do not add medians from separate runs. | EvidenceSpeech-to-speech boardConversation latency test |
How should production latency be measured?
- 01Timestamp the last caller audio, committed transcript, first LLM token, first TTS byte and first played audio.
- 02Run the same prompts, tools, voices, regions and network path for every candidate.
- 03Publish sample size, run date, median and tail latency. Keep errors and empty responses in the denominator.
- 04Replay pauses, backchannels, noisy audio and tool calls from the calls the agent must actually handle.
What usually moves the number?
- Endpointing can add delay before any model request begins.
- LLM tokens per second do not answer the first-token question.
- TTS must receive a speakable chunk before it can stream audio.
- Region placement and media playback add time outside the model APIs.
- A model or provider change should be evaluated on the full call, not a single component score.
Cascade or speech-to-speech?
A cascade exposes separate STT, LLM and TTS clocks and lets the application replace one component. A speech-to-speech model can shorten the path, but it changes the observability, cost and component-control tradeoff. Compare both on the same complete conversation before changing the architecture.
Read the measured cascade and speech-to-speech comparison, then inspect the voice agent infrastructure map.
Measure the complete turn, then change one stage at a time.