It's 2:13am. A potential customer calls your SaaS support line. Your AI agent answers. They ask a question. Then silence. One second. Two seconds. They hang up. You just lost $4,800 in annual contract value to a timeout.
This isn't a hypothetical. We see it in our logs every week. Text-based chat widgets have trained everyone to accept latency. You type a message, you wait. Maybe you open another tab. It's asynchronous by nature.
Voice is synchronous. When someone speaks to you, you respond in under 300ms or the conversation feels broken. Google found that 250ms of added latency in search results drops user satisfaction measurably. In voice? The threshold is even tighter.
The Three Latency Layers in Voice AI
Most founders optimizing voice agents focus on the wrong layer. There are actually three places latency hides:
- Speech-to-text (STT): How fast your system transcribes the customer's words. OpenAI Whisper large-v3 averages 180-240ms on streaming mode. Deepgram Nova-2 gets it down to 90-140ms.
- LLM inference: How fast your agent "thinks" and generates a response. GPT-4 Turbo starts streaming tokens in 400-600ms. Claude 3.5 Sonnet is 200-350ms. Llama 3.3 70B on dedicated metal can hit 150ms.
- Text-to-speech (TTS): How fast your system turns the response into audio. OpenAI tts-1 with the nova voice starts streaming audio in 120ms. ElevenLabs turbo v2.5 is around 180ms. Azure Neural voices are 250-400ms.
Add network round-trips, and you're looking at 700ms to 1,400ms in the best case. That's the gap between "this feels like talking to a person" and "this feels like talking to a machine."
Why Text Chatbots Get Away With Slow, Voice Agents Don't
Text chat is forgiving because it's visual. You see a typing indicator. You see the message being composed token by token. Your brain stays engaged because there's constant feedback.
In voice, there's no typing indicator. There's either sound or silence. Silence for more than 400ms triggers discomfort. Silence for more than 1 second triggers "is this thing broken?"
That's why Intercom, Drift, Tidio, Crisp—none of them have shipped real voice. It's not because they can't wire up Whisper and GPT-4. It's because voice exposes latency in a way text never does, and they haven't architected for it.
"In a voice conversation, 500ms isn't a technical metric. It's the moment your customer decides whether they're talking to something smart or something stupid."
How We Keep Softnode Agents Under 400ms End-to-End
We treat latency as a product feature, not an infrastructure detail. Here's the stack:
- Streaming STT: We use Deepgram Nova-2 with endpointing tuned to 0.8 seconds of silence. It starts transcribing before the customer finishes their sentence.
- Parallel LLM calls: For common intents ("What are your hours?" "How much does it cost?"), we pre-generate responses and cache them per agent. For dynamic queries, we use Claude 3.5 Sonnet with a 200-token max output limit and streaming enabled.
- Streaming TTS: We use OpenAI tts-1 with the nova voice. It starts playing audio before the LLM finishes generating the full response. The customer hears the first word in under 300ms from when they stop speaking.
- Edge routing: Our agents run on Cloudflare Workers in 300+ locations. If you're calling from Istanbul, you hit an Istanbul edge. If you're in Chicago, you hit Chicago. No cross-Atlantic round trips.
The Business Case for Fast Voice
Low latency isn't just a better experience—it's a conversion lever. We ran a cohort analysis on 2,400 inbound voice calls to SaaS demos in March. Calls where the agent responded in under 400ms had a 34% higher booking rate than calls where the agent took 800ms+.
Why? Because fast responses signal competence. If your AI agent sounds slow, the customer assumes your product is slow. If your agent sounds sharp, they assume your product is sharp.
For a hair transplant clinic in Istanbul taking 60 inquiries a week, a 34% lift in bookings is 20 more consultations a month. At a €3,500 average procedure price and 40% close rate, that's €28,000 in monthly revenue. From latency optimization.
What to Do Right Now
If you're building or buying a voice AI agent, measure latency first. Most vendors don't publish it. Most demos hide it with clever UX. Ask for P50 and P95 end-to-end response time, measured from end-of-speech to start-of-audio.
If you're using a text-only widget (Intercom, Drift, Crisp, Tidio), understand that adding voice later isn't a feature flag. It's a re-architecture. Those platforms weren't built for sub-500ms response loops.
If you're evaluating Softnode, test it with a real question at 2am. Not a demo script. A real question your customers ask. Listen for the gap. If it feels like a conversation, it's working. If it feels like waiting, it's not.
Latency is the difference between a voice agent that gets used and a voice agent that gets muted. We built Softnode to win on that difference.
Want a voice agent that responds in under 400ms?
Softnode agents speak 23 languages, book meetings, answer questions, and feel like talking to a real person. No six-month implementation. No vendor lock-in. Just fast, natural voice AI.
Try Softnode Free