vLLM v0.34.0新增10个模型并默认启用可扩展KV缓存
AI 总结
vLLM v0.34.0新增Mistral Large 4、MiniMax-M3.1-Flash和NVIDIA Nemotron-3.5-Transcribe等10个模型,并默认启用可扩展KV缓存。此次发布移除Model Runner V1,要求使用huggingface_hub 2.0.0及以上版本,将默认块大小改为64,并停止提供ROCm 7.2构件。vLLM称,在Blackwell和MI355X上,解码速度最高提升1.95倍、吞吐量提升1.6倍,长提示词TTFT加快1.24至1.40倍。
深度见解
这是AI推理引擎的重要更新,扩展了模型支持并调整多项关键运行时默认值,但Blackwell和MI355X上的性能提升仍属于厂商声明。
v0.34.0
1,061 commits from 412 contributors (120 new). Thank you!
TL;DR
- 10 new models (Mistral Large 4, MiniMax-M3.1-Flash, Nemotron-3.5-Transcribe, and more!)
- Extensible KV cache is on by default: vLLM sizes the KV cache from memory actually free after warmup, so most models get a bigger cache and more throughput - in most cases you should now remove
gpu_memory_utilizationfrom your configs. - Model Runner V1 is gone. Model Runner V2 is now the only GPU runner.
- Big speedups on Blackwell and MI355X: up to 1.95× decode (Step5 on GB200), 1.6× throughput (GLM-5.3-Flash), 1.24–1.40× faster long-prompt TTFT (GLM-5.3).
- Upgrade check: default block size is now 64, ROCm 7.2 images are gone, and
huggingface_hub >= 2.0.0is required. See Before you upgrade.
Before you upgrade
Important
部分来源内容未展示,可前往原文继续阅读。
#vLLM#Mistral Large 4#MiniMax-M3.1-Flash#NVIDIA Nemotron-3.5-Transcribe#Ricursive Ripple-65B Preview#Zyphra ZAYA1#Baidu Unlimited-OCR-2#Liquid AI LFM3#NVIDIA Canary-1B-v2#Ming3-ASR#NVIDIA Parakeet TDT