SGLang报告追平或超过DeepInfer
AI 总结
DeepSeek开源了自家服务背后的推理引擎DeepInfer及相关内核。根据SGLang项目方的报告，其使用DeepInfer基准和相同硬件开展的测试显示，SGLang在4张GB300或4张B300上的预填充吞吐最高达到DeepInfer的1.04倍，在64张GB300上的解码单步耗时最多缩短7.4%。36氪还报道称，vLLM在各自披露的测试配置下已接近或部分超过DeepInfer。这些结果来自项目方自测，比较所用的工作负载和测量方法存在差异。
深度见解
报道显示，DeepInfer开源后，SGLang和vLLM吸收了其技术并继续优化。SGLang还提供DeepInfer此次发布未包含的服务组件，包括API服务器、KV缓存、预填充与解码分离以及副本间负载均衡。
On this page
- What is integrated
- Results
- Where the speed comes from
- What SGLang adds
- Acknowledgments
- Appendix: benchmark setup
- Prefill
- Decode
TL;DR
- On par or ahead on DeepInfer’s own benchmarks. Up to 1.04x prefill throughput on 4x GB300 and 4x B300, up to 7.4% shorter decode steps on 64x GB300, and within 1% at the remaining points.
- DeepInfer’s optimizations, in SGLang. DeepSeek’s kernels including the Mega series, DSpark speculative decoding, SWA bounded replay and both Engram placements run in SGLang main.
- A full serving system. SGLang is an API server, and it has the KV cache system, prefill/decode disaggregation and load balancing across replicas that DeepInfer’s README lists as not included in its release.
#DeepSeek#DeepInfer#SGLang#vLLM#DeepSeek-V4.1-Flash#NVIDIA GB300#NVIDIA B300#FlashMLA#DeepGEMM#DeepEP#DSpark#DeepInfer kernels