SemiAnalysis AgentX

Headlines · 1

1. Oct 23
   vLLM Reports 954K Tokens per Second per GPU for DeepSeek V4.1 Flash

   In a vLLM-published benchmark, DeepSeek V4.1 Flash reached 954,000 tokens per second per GPU on a single NVIDIA GB300 NVL72 rack using P/D disaggregation and Wide-EP, with P90 throughput of 196 tokens per second per user. On GB200, vLLM’s decode throughput was within 6% of DeepInfer’s published results across batch sizes 8–256. A specialized decode path achieved 826 P90 tokens per second for one user on four B300 GPUs. In an AgentX prefill test at concurrency 256, DEP2 delivered 64,400 tokens per second per GPU, compared with 53,300 for DEP4. The report said the configurations differed and did not characterize the difference as an engine-to-engine speedup; replay tests attributed a 13.6% DEP4 throughput loss to data-parallel imbalance.

   vLLM Blog · 🔥 0

Experience and discussion from the community

Share my SemiAnalysis AgentX experienceAsk about SemiAnalysis AgentX