vLLM v0.34.0 adds 10 models and enables extensible KV cache by default
vLLM v0.34.0 introduces 10 models, including Mistral Large 4, MiniMax-M3.1-Flash and NVIDIA Nemotron-3.5-Transcribe, and makes extensible KV caching the default. The release removes Model Runner V1, requires huggingface_hub 2.0.0 or later, changes the default block size to 64 and drops ROCm 7.2 artifacts. vLLM reports up to 1.95× faster decoding, 1.6× higher throughput and 1.24–1.40× faster long-prompt TTFT on Blackwell and MI355X hardware.
This is a significant inference-engine update that expands model support and changes important runtime defaults, but its reported Blackwell and MI355X performance gains remain vendor claims.
v0.34.0
1,061 commits from 412 contributors (120 new). Thank you!
TL;DR
- 10 new models (Mistral Large 4, MiniMax-M3.1-Flash, Nemotron-3.5-Transcribe, and more!)
- Extensible KV cache is on by default: vLLM sizes the KV cache from memory actually free after warmup, so most models get a bigger cache and more throughput - in most cases you should now remove
gpu_memory_utilizationfrom your configs. - Model Runner V1 is gone. Model Runner V2 is now the only GPU runner.
- Big speedups on Blackwell and MI355X: up to 1.95× decode (Step5 on GB200), 1.6× throughput (GLM-5.3-Flash), 1.24–1.40× faster long-prompt TTFT (GLM-5.3).
- Upgrade check: default block size is now 64, ROCm 7.2 images are gone, and
huggingface_hub >= 2.0.0is required. See Before you upgrade.
Before you upgrade
Important
Some source content is omitted. Continue reading on the original site.
#vLLM#Mistral Large 4#MiniMax-M3.1-Flash#NVIDIA Nemotron-3.5-Transcribe#Ricursive Ripple-65B Preview#Zyphra ZAYA1#Baidu Unlimited-OCR-2#Liquid AI LFM3#NVIDIA Canary-1B-v2#Ming3-ASR#NVIDIA Parakeet TDT