Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

AWS shows sentence-by-sentence TTS streaming for voice agents with Kokoro-82M

AI summary

An AWS Machine Learning Blog post walks through streaming each sentence an AI agent generates to a hosted Kokoro-82M text-to-speech model on Amazon SageMaker AI, so synthesis starts on sentence one instead of waiting for the finished response. The post frames the problem as a latency gap: a one-sentence response might generate in 600 milliseconds while a full response takes five seconds, during which the customer hears nothing. Kokoro-82M's 82 million parameters run in a custom container on an ml.g5.xlarge real-time endpoint with two routes, a POST streaming route and a bidirectional WebSocket route; AWS also compares returning audio to a browser over Server-Sent Events, where base64 encoding inflated payloads by close to the expected 33 percent and MP3 frames did not align with delivery chunks. AWS reports time to first audio fell by more than half in isolation and by 2.4 to 3.5 times in a deployed agent, and notes the handlers run a strictly sequential loop rather than concurrent synthesis.

In-depth insight

The walkthrough gives teams adding voice to text-streaming agents a concrete pattern, and quantified costs, for starting speech synthesis before an agent's full answer is generated.

A customer asks an AI voice agent a question and waits. The agent already has an answer, it’s midway through generating a long, multi-paragraph response, but the customer hears nothing for five seconds. By the time an answer arrives, the customer is frustrated, confused, and no longer paying attention.

Text-to-speech integrations create exactly that gap. A one-sentence response might generate in 600 milliseconds, but the full response takes five seconds, and the integration waits for all of it before synthesis starts.

This post shows how to close that gap. We stream each sentence to a hosted Kokoro-82M text-to-speech (TTS) model on Amazon SageMaker AI as soon as an agent produces it, so synthesis begins on sentence one instead of the finished response. Getting that audio back to a browser needs its own solution too, so we also cover both transports for doing it and what each costs in practice.

Solution overview

This solution closes the gap by streaming synthesis alongside generation instead of waiting for it to finish. As the agent produces each sentence, that sentence streams immediately to a hosted Kokoro-82M text-to-speech model, so audio playback can start before the agent finishes speaking its full response. Kokoro-82M holds only 82 million parameters, small enough to run cheaply on a single mid-tier GPU instance. That keeps both latency and inference cost low for a voice agent handling real traffic.

#AWS#Amazon SageMaker AI#Kokoro-82M#Amazon Bedrock AgentCore#Amazon Transcribe

Original · 12 min readRead the original
Group chat

No comments yet. Start the conversation.