Baseten称Qwen3-ASR批处理转录吞吐量提升7倍
Baseten 称，其在单张 NVIDIA RTX PRO 6000 上优化了 Qwen3-ASR 1.7B 部署，采用原生音频解码、FP8 KV 缓存量化和减少主机与设备间同步等方法。在高并发条件下，优化后的技术栈每秒处理音频量最高达到原生 vLLM 的 7 倍。Baseten 还称，在处理一小时音频文件时可持续实现超过 3,000 倍实时速度，估算每音频小时成本约为 0.0013 美元，字错误率没有明显变化。Baseten 称 Qwen3-ASR 1.7B 位居 Hugging Face Open ASR Leaderboard 开放模型首位，平均字错误率为 5.4%。这些部署性能及成本数据均由厂商披露。
该报告表明，在不更换底层开放权重模型的情况下，解码、KV 缓存量化和同步机制方面的优化可显著提升批量语音转文本的吞吐量并降低估算成本。
Published
October 28, 2026
TL;DR
Qwen3-ASR 1.7B is the top-ranking open-source model on HuggingFace's Open ASR Leaderboard, but serving it efficiently for batch transcription required careful optimization. We combined native audio decoding, KV cache quantization, and reduced unnecessary host-to-device syncs to serve 7x more audio per second at high concurrency on a single RTX PRO 6000, sustaining over 3,000x real time on hour-long files at about a tenth of a cent per audio hour, with no meaningful change in word error rate.
Speech-to-text splits into two very different workloads. Realtime transcription is about latency: a voice agent needs words back within a few hundred milliseconds of someone speaking. Batch transcription is about throughput: podcast networks, meeting-notes products, and call-center archives have hours of recorded audio and want it transcribed as cheaply as possible. That means getting as many hours of audio through each GPU as it can handle.
#Baseten#Qwen3-ASR 1.7B#Qwen#vLLM#NVIDIA RTX PRO 6000#Hugging Face Open ASR Leaderboard