vLLM报告DeepSeek V4.1 Flash每GPU吞吐达95.4万tokens/s
在一项由vLLM发布的基准测试中，DeepSeek V4.1 Flash使用P/D分离和Wide-EP，在单个NVIDIA GB300 NVL72机架上达到每GPU 95.4万tokens/s，单用户P90吞吐为196 tokens/s。在GB200上，vLLM的解码吞吐在批量8至256时与DeepInfer公开结果的差距不超过6%。专用解码路径在四张B300上为单用户实现826 P90 tokens/s。在并发256的AgentX预填充测试中，DEP2达到每GPU 6.44万tokens/s，DEP4为5.33万。报告指出两套配置不同，因此没有将曲线差距解释为引擎之间的加速比；回放测试显示，DEP4有13.6%的吞吐损失来自数据并行失衡。
该研究披露了在NVIDIA基础设施上服务大型MoE模型的性能数据与优化经验，包括预填充阶段采用较小DEP实例，以及面向低延迟解码的专用路径。核心基准结果来自vLLM发布的单项研究，尚未得到独立验证。
TL;DR: On one NVIDIA GB300 NVL72 rack, vLLM with P/D disaggregation and Wide-EP serves DeepSeek-V4.1-Flash on SemiAnalysis AgentX at 954K tokens/s per GPU at a P90 of 196 tokens/s per user, 7.9× our single-node result from the first post and $38.96 per GPU-hour at DeepSeek's off-peak prices. A specialized decode path reaches 826 tokens/s for a single user. Against DeepInfer, DeepSeek's own engine, vLLM's decode throughput is within 6% across batch sizes 8–256, and its prefill tracks DeepInfer's published curve.
The efficiency comes from deliberate engineering on components that vLLM already has, built on the kernels DeepSeek open-sourced. This post walks through our performance study, breaks down each optimization, and shares the lessons we learned:
- A general-purpose engine can match a model-specific one
- Understand the model before optimizing
- Small DEP instances work best for prefill1
- Wide-EP decode scales well with EPLB
- CPU overhead becomes critical at NVL72 scale
- A specialized decode path unlocks low latency
#DeepSeek V4.1 Flash#vLLM#DeepInfer#NVIDIA GB300 NVL72#NVIDIA B300#SemiAnalysis AgentX#Dynamo