DeepSeek-V4.1-Flash Serving Optimized on H20-3e
AI summary
A blog reported that tuning prefill and decode parallelism, using FP8 DeepGEMM scoring, improving CUDA graphs and optimizing Indexer kernels reduced mean end-to-end latency by 65.5% and produced 2.91 times the output throughput of a TP4 baseline in the authors’ H20-3e tests. Mean time to first token fell from 12.04 seconds to 1.89 seconds, while mean time per output token after the first rose from 3.20 to 5.62 milliseconds. A four-GPU colocated configuration reduced mean latency by 57.26% and increased throughput by 130.79% against its BF16 TP4 baseline. The report includes a public SGLang branch, a specific commit and environment instructions for reproduction.
On this page