Baseten Reports 7x Throughput Boost for Qwen3-ASR Batch Transcription
Baseten reports that it optimized a Qwen3-ASR 1.7B deployment on a single NVIDIA RTX PRO 6000 using native audio decoding, FP8 KV-cache quantization, and fewer host-to-device synchronizations. At high concurrency, the optimized stack processed up to 7 times more audio per second than out-of-the-box vLLM. Baseten also reports sustaining more than 3,000 times real time on hour-long files, at an estimated cost of about $0.0013 per audio hour, with no meaningful change in word error rate. It describes Qwen3-ASR 1.7B as the top-ranking open-source model on Hugging Face's Open ASR Leaderboard, with a 5.4% average word error rate. These deployment performance and cost figures are vendor-reported.
The report shows how decoding, KV-cache quantization, and synchronization changes can substantially improve batch speech-to-text throughput and estimated cost without changing the underlying open-weight model.
Published
October 28, 2026
TL;DR
Qwen3-ASR 1.7B is the top-ranking open-source model on HuggingFace's Open ASR Leaderboard, but serving it efficiently for batch transcription required careful optimization. We combined native audio decoding, KV cache quantization, and reduced unnecessary host-to-device syncs to serve 7x more audio per second at high concurrency on a single RTX PRO 6000, sustaining over 3,000x real time on hour-long files at about a tenth of a cent per audio hour, with no meaningful change in word error rate.
Speech-to-text splits into two very different workloads. Realtime transcription is about latency: a voice agent needs words back within a few hundred milliseconds of someone speaking. Batch transcription is about throughput: podcast networks, meeting-notes products, and call-center archives have hours of recorded audio and want it transcribed as cheaply as possible. That means getting as many hours of audio through each GPU as it can handle.
#Baseten#Qwen3-ASR 1.7B#Qwen#vLLM#NVIDIA RTX PRO 6000#Hugging Face Open ASR Leaderboard