Practical guide. Provider claims are attributed to the sources below. This page does not claim a new independent DARC benchmark.
For: Teams building voice assistants or speech-enabled products
Before you start: Permission to use test recordings · A small set of expected task outcomes
⚡ TL;DR — the 30-second version
- Test corrections, interruptions and exact identifiers.
- Separate transcript, reasoning and action failures.
- Do not confuse speaking cancellation with cancelling an external action.
Voice systems fail across several layers: audio capture, recognition, reasoning, tools and speech output. A pleasant voice or low word error rate alone cannot prove that a booking assistant schedules the right appointment. Evaluate the user-visible task and the intermediate stages.
The current stack includes native audio models such as Gemini 3.8 Live and modular pipelines such as ElevenLabs Speech Engine around a separate text model. Those designs have different latency, interruption and observability trade-offs.
“A fluent voice can make a wrong action harder to notice. Evaluate task correctness before voice preference.”
Field guide / 01
01 / Build difficult but realistic audio cases
Include proper names, dates, currencies, account numbers, accents, background noise and long pauses. Test corrections such as 'not Tuesday, Thursday' and requests that change while a tool is running. Use consented or synthetic recordings and label their expected interpretation.
Keep the same cases across providers, and separate recognition mistakes from reasoning mistakes. If the transcript is wrong, debugging the prompt may hide the real problem.
| Scenario | Expected behavior |
|---|---|
| User interrupts speech | Stop speaking and cancel or safely reconcile in-flight work |
| User corrects a date | Confirm the revised date before booking |
| Tool is still running | Acknowledge progress without claiming completion |
| Unclear identifier | Ask for clarification instead of guessing |
| User withdraws consent | Stop the requested recording or processing workflow |
Field guide / 02
02 / Measure the complete timeline
Track end of user speech, final transcript, first model output, tool completion and first audible response. Measure distributions, not one best-case recording. Long-tail pauses are often more disruptive than a small average improvement.
Asynchronous tools can keep a conversation flowing, but the assistant must distinguish 'I am checking' from 'it is done'. The Gemini Live documentation explicitly supports background work; your UX still needs an honest completion signal.
Field guide / 03
03 / Make actions confirmable
Repeat high-stakes details in a concise confirmation before executing. A read-back of a booking date and recipient is more useful than asking an ambiguous 'is that okay?' Separate voice consent, account authorization and permission for the specific action.
Test cancellation after a request has left your system. Cancelling speech or model inference does not necessarily cancel an external booking. Use operation IDs and reconciliation so the interface can report the real state.
Field guide / 04
04 / Budget by the right units
Speech services may bill minutes, characters, tokens or optional recognition features. The reviewed Scribe documentation lists additional charges for some features; Gemini Live's developer announcement describes per-minute audio pricing. A text-token calculator alone will miss these categories.
Report cost per completed conversation, accepted task and usable minute alongside latency and review burden. Keep voice-preference ratings separate from objective task outcomes.
Before you ship
0/5Use this as a review aid. Checks reset when you leave this page.
Sources & verification
Last checked 23 Sept 2026. Publication dates and access conditions belong to the linked originals.
- 01Gemini 3.8 Live and Live Extended ThinkingGoogle↗
- 02Gemini 3.8 Live model specificationsGoogle↗
- 03Building with Gemini Audio — per-minute pricingGoogle↗
- 04ElevenLabs — Speech EngineElevenLabs↗
- 05ElevenLabs — Scribe v2 and optional featuresElevenLabs↗
