SGLang Reports Matching or Beating DeepInfer
DeepSeek open-sourced DeepInfer, the inference engine behind its own service, along with related kernels. In project-run tests using DeepInfer’s benchmarks and the same hardware, SGLang reached up to 1.04x prefill throughput on four GB300 or four B300 GPUs and cut decode step time by as much as 7.4% on 64 GB300 GPUs, according to the SGLang report. A 36氪 report said vLLM also approached or partly exceeded DeepInfer under its disclosed configurations. The comparisons used project-reported results and differing workloads and measurement methods.
The reports describe post-release integration of DeepSeek techniques by SGLang and vLLM. SGLang also provides serving components that DeepInfer’s release did not include, including an API server, KV caching, prefill/decode disaggregation and load balancing across replicas.
On this page
- What is integrated
- Results
- Where the speed comes from
- What SGLang adds
- Acknowledgments
- Appendix: benchmark setup
- Prefill
- Decode
TL;DR
- On par or ahead on DeepInfer’s own benchmarks. Up to 1.04x prefill throughput on 4x GB300 and 4x B300, up to 7.4% shorter decode steps on 64x GB300, and within 1% at the remaining points.
- DeepInfer’s optimizations, in SGLang. DeepSeek’s kernels including the Mega series, DSpark speculative decoding, SWA bounded replay and both Engram placements run in SGLang main.
- A full serving system. SGLang is an API server, and it has the KV cache system, prefill/decode disaggregation and load balancing across replicas that DeepInfer’s README lists as not included in its release.
#DeepSeek#DeepInfer#SGLang#vLLM#DeepSeek-V4.1-Flash#NVIDIA GB300#NVIDIA B300#FlashMLA#DeepGEMM#DeepEP#DSpark#DeepInfer kernels