Humming 0.1.18 expands low-bit MoE integration in SGLang
Humming 0.1.18 expands the SGLang integration across more quantization schemes and GPU architectures, including Hopper and Blackwell. The project also introduced a shared activation-expression interface for SGLang's MoE runners. In its DeepSeek-V4.1-Flash-1017 evaluation on eight H20-3e GPUs, Humming A16 achieved project-reported prefill speedups of 1.07–1.23× over Marlin A16 at the same precision in the tested long-context cases. The report also covers FP8 expert activations, speculative-decoding acceptance and quality checks limited to the stated EP1 configurations.
The release provides SGLang users with a unified integration for quantized dense and MoE inference across supported quantization schemes and GPU architectures, alongside project-reported serving evaluations and configuration-specific quality checks.
On this page
- One Backend Across Quantized Representations
- Prepare Once, Specialize for the Workload
- Hopper: Compact MXFP4 Storage, FP8 Computation
- Blackwell: Additional Native Compute Paths
- Connecting Humming to SGLang
- Using Humming in SGLang
- Serving Results on H20-3e
- Prefill
- Decode
- Understanding the Results
- Benchmark Details
- What Comes Next
- Humming
- SGLang Integration
- Acknowledgments
#Humming 0.1.18#SGLang#Ant Group#DeepSeek-V4.1-Flash-1017#Marlin#FlashInfer 0.7.2#NVIDIA Hopper#NVIDIA Blackwell#NVIDIA H20-3e