Voice agent infrastructure for STT, LLM and TTS

Voice agent infrastructure, also called voice AI infrastructure, is the realtime stack that carries a spoken interaction from telephony or WebRTC through turn detection, speech-to-text, an LLM, text-to-speech and observability, then back to the caller. A voice model router is one layer in that stack: it lets a team select, pin, observe and replace models without rebuilding the runtime around each provider.

Speko gives an existing LiveKit, Pipecat or OpenAI SDK pipeline one base URL and bearer key for its model calls. The runtime still owns the conversation; Speko selects or pins an eligible model before the turn. On a successful model call, response headers report the serving route.

Get API key Read the quickstart Browse benchmarked models

Where does voice agent infrastructure sit?

Caller and transport

Audio enters through telephony or WebRTC.

Agent runtime

Turns, VAD, context and tools stay in the application.

Speko router

A compatible request is constrained, selected or pinned.

Model provider

The chosen STT, LLM or TTS model serves the request.

Layer	What it owns	Relationship to Speko
Transport and telephony	Calls, WebRTC rooms, media transport and phone numbers.	Keep it. Speko receives model calls; it does not carry the call.
Agent runtime	Turn detection, VAD, context, tools and the realtime control loop.	Keep LiveKit, Pipecat or your own runtime and change its model client.
Voice model router	Selection, pinning and pre-response failover across STT, LLM and TTS.	Speko is this layer: one base URL and bearer key for the three stages.
Hosted agent platform	Prompt, voice, tools, phone number and deployment as one managed product.	Optional. Use a platform when you do not want to operate the runtime.

Speko also runs the whole agent: the prompt, the voice, its tools and its phone number live in the console. The router and hosted platform are separate ways to use Speko; a router key does not replace the agent runtime.

How is Speko different from OpenRouter for voice agents?

As of August 2, 2026, the overlap is real. OpenRouter publishes dedicated speech-to-text and text-to-speech APIs, audio input and output through Chat Completions, an automatic LLM router, cross-model fallbacks and provider routing. Speko does not claim those capabilities are absent. Its narrower distinction is automatic selection at each voice stage from published, dated voice measurements, while the existing runtime keeps the realtime loop.

Area	OpenRouter	Speko
Voice API surface	Dedicated speech-to-text and text-to-speech endpoints, plus audio input and output through Chat Completions. Audio STT TTS	Compatible speech-to-text, LLM and text-to-speech routes behind one base URL.
Automatic model selection	openrouter/auto-beta selects an LLM for the prompt; openrouter/auto is deprecated. With an explicit models list, provider.sort.partition set to "none" can sort across model boundaries by price, throughput or latency. Auto Router Provider routing	model: "auto" ranks an eligible model independently at STT, LLM and TTS from published voice benchmark rows.
Application boundary	The application or agent framework still owns the realtime conversation loop. LiveKit	LiveKit, Pipecat or a custom runtime still owns the realtime conversation loop.
Decision evidence	On Auto Router completion routes, the response model field identifies the selected model. Detailed completion-router metadata is available only when the request opts in with X-OpenRouter-Metadata: enabled; dedicated STT and TTS responses use their speech-specific shapes. Auto Router Router metadata STT TTS	Response headers identify the route and reason; the voice measurements are public and dated.

OpenRouter's LiveKit guide and Speko's LiveKit guide both leave conversation control with the runtime. The difference is which model-selection evidence and voice stages sit behind the model calls.

How does measured routing work?

Constrain the request

Send model "auto" with an enabled language and objective. The production router accepts English and Spanish, including regional tags such as en-US and es-MX; other primary languages are rejected. Allow, deny and maximum-price controls can narrow the eligible set. To pin a route, use the id from a GET /v1/models row whose routable field is true.
Rank benchmark candidates

Automatic routing considers only models in the published benchmark snapshot. If the request constraints leave no eligible candidate, the router returns no_candidate rather than selecting outside that snapshot.
Serve the selected route

The compatible STT, LLM or TTS endpoint forwards the request to the selected provider. X-Route means: Serving provider and model on success; last attempted route on terminal upstream failure, or none/none if no route was attempted.
Inspect before production traffic

A preview endpoint returns the provider Speko would pick for a language and objective, with its runners-up, before you send a request.

Published router surface

Captured 2026-08-05 from the public model catalog. The combination count is one routable model from each stage: 14 x 15 x 15 = 3,150. It is not a distinct-model count.

Stage	Endpoint	Routable
Speech-to-text	/v1/audio/transcriptions	14 of 15
LLM	/v1/chat/completions	15 of 16
Text-to-speech	/v1/audio/speech	15 of 17

Public evidence

Speko publishes these first-party results from dated benchmark runs and listening panels; they are not live provider telemetry or an independent study. The conditions below come from the checked-in snapshot used by this build's model pages. The linked public boards are mutable and can show a newer run.

Stage	Published run conditions
Speech-to-text	English ; read speech (FLEURS, n=50) ; batch 2026-07-03, streaming 2026-07-21
LLM	English ; behavior measured 2026-07-17 (n=3 iters) ; TTFT/cost from us-east4
Text-to-speech	Naturalness and win % / en-US, blind human panel, no brand labels / 15 systems / bootstrap 95% CI / panel July 2026
Full stacks	dental-booking task ; n=3/stack ; grounded (tool fired) ; measured 2026-07-03

Inspect this build's content-addressed benchmark snapshot (SHA-256 540a3c309f857c71ee4ce6c25f1afda5bd024c8de7111680abcdf617109d1605) or the mutable publisher JSON for board dates, metric definitions, model rows and row-level rerun notes. The snapshot preserves the exact transformed rows and source receipt used by this page. The public GET /v1/models catalog is narrower: it lists the integrated STT, LLM and TTS rows carried by the deployed Router, including each row's routable flag. The benchmark catalog intentionally retains additional benchmark-only measured rows for comparison; appearing there does not make a row a Router catalog entry or an eligible selection or pin. Rows marked routable: false stay visible as measurements but are excluded from automatic selection and cannot be pinned.

How do you add it to an agent?

Point an OpenAI client at Speko, keep model: "auto" for measured selection. Compare candidates in the benchmark catalog. To pin a route, use the id from a GET /v1/models row whose routable field is true. The same client shape covers the three cascade stages; this example is the text-to-speech call, and the quickstart includes the matching transcription and chat calls.

API quickstart LiveKit guide Pipecat guide

import OpenAI from "openai";

const speko = new OpenAI({
  apiKey: process.env.SPEKO_API_KEY,
  baseURL: "https://api.speko.ai/v1",
  defaultHeaders: {
    "X-Speko-Language": "en",
    "X-Speko-Objective": "latency",
  },
});

const speech = await speko.audio.speech.create({
  model: "auto",
  voice: "alloy",
  input: "We close at six on Saturday.",
  response_format: "pcm",
});

What happens when a provider fails?

Failover is pre-response only. For replayable requests, Speko can try another eligible route after a connection or upstream-body failure, a first-byte timeout, a redirect, a provider 401, 403, 404, 408 or 429 response, or a 5xx response - but only before a downstream byte has been sent. Multi-step adapters can classify provider failures differently.

The public OpenAPI contract documents the route, failover-count and first-byte response headers plus the terminal upstream-failure response. The API reference separately provides request shapes and response-header and error tables.

Once audio or text is streaming, the request stays on that provider or terminates with it. Speko does not move an active stream or WebSocket mid-call; a route change applies only to a new request before its stream begins.

A failed upstream attempt opens a short provider circuit. Later requests bypass the open circuit until its backoff expires, when one request probes recovery. This is request-driven recovery, not an active synthetic monitor or health dashboard.

A request can be replayed only when the Router buffered it at or below the configured cap, 8 MiB by default. The other compatible POST routes reject a nonreplayable body with 413; for POST /v1/audio/transcriptions, that includes a chunked upload with no declared content length. POST /v1/transcribe is the raw-stream exception: an upload with no declared content length, or one that exceeds the cap, is nonreplayable and only one available candidate is attempted. That route can proceed only if it can forward the upload without replayable-body request translation.

Current limitation: translation-dependent HTTP STT adapters are not all removed before that ranking. An automatic selection can therefore choose one, fail while preparing the request and return 502 instead of trying another provider. A streamed attempt never fails over, and X-Route-Reason includes nonreplayable-body.

The application still owns operating conditions

Speko does not infer that a call is noisy, mobile or landline. If those conditions should change the route, the application must detect them and apply an allow, deny, objective or model choice. Speko also does not include an A/B experiment dashboard; record the serving headers in your own analytics.

Headers that ship today

X-Route	Serving provider and model on success; last attempted route on terminal upstream failure, or none/none if no route was attempted.
X-Route-Reason	Compact explanation of the routing decision
X-Speko-Failover-Count	Recorded failed or circuit-bypassed candidates before success, or before a terminal failure was returned.
X-Speko-First-Byte-Ms	Router-observed elapsed time at the selected adapter boundary. Successful direct HTTP routes include the first upstream body frame; buffered or translated adapters can include the completed upstream operation; terminal failures can omit this header.

References: public OpenAPI and API reference.

When is a direct provider simpler?

Use a voice model router when

You need to compare or replace providers without changing every integration.
Different languages, objectives or price limits need different eligible routes.
You want a serving receipt and pre-response failover behind one compatible API.

Call a provider directly when

One fixed model is enough and you do not need cross-provider selection.
You need a provider-native feature that a compatible surface does not expose.
You prefer to own credentials, measurement and retry policy in the application.

Review the routing price, inspect the current model surface, and run a narrow pilot against the providers you already use before changing production traffic.

Use the voice agent latency measurement framework to keep the end-to-end and component clocks comparable. If the open question is who should own the realtime loop, telephony or model layer, start with the sourced platform, runtime and router comparison.