NVIDIA H20-3e
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 2
- DeepSeek-V4.1-Flash Serving Optimized on H20-3e
A blog reported that tuning prefill and decode parallelism, using FP8 DeepGEMM scoring, improving CUDA graphs and optimizing Indexer kernels reduced mean end-to-end latency by 65.5% and produced 2.91 times the output throughput of a TP4 baseline in the authors’ H20-3e tests. Mean time to first token fell from 12.04 seconds to 1.89 seconds, while mean time per output token after the first rose from 3.20 to 5.62 milliseconds. A four-GPU colocated configuration reduced mean latency by 57.26% and increased throughput by 130.79% against its BF16 TP4 baseline. The report includes a public SGLang branch, a specific commit and environment instructions for reproduction.
LMSYS Blog · 🔥 4 - Humming 0.1.18 expands low-bit MoE integration in SGLang
Humming 0.1.18 expands the SGLang integration across more quantization schemes and GPU architectures, including Hopper and Blackwell. The project also introduced a shared activation-expression interface for SGLang's MoE runners. In its DeepSeek-V4.1-Flash-1017 evaluation on eight H20-3e GPUs, Humming A16 achieved project-reported prefill speedups of 1.07–1.23× over Marlin A16 at the same precision in the tested long-context cases. The report also covers FP8 expert activations, speculative-decoding acceptance and quality checks limited to the stated EP1 configurations.
LMSYS Blog · 🔥 0
Experience and discussion from the community
Share my NVIDIA H20-3e experienceAsk about NVIDIA H20-3e
Nobody has shared their experience with NVIDIA H20-3e yet.