DeepSeek Opensources DeepInfer; LMSYS Benchmarks SGLang Against It
DeepSeek has open-sourced DeepInfer, the inference engine behind its service, together with kernels used by the system. LMSYS tested SGLang against DeepInfer using DeepInfer’s benchmarks on the same hardware, reporting up to 1.04x prefill throughput on four NVIDIA GB300 or four B300 GPUs and decode steps up to 7.4% shorter on 64 GB300 GPUs. The comparison is not fully equivalent because request-token patterns, averaging methods, Engram table placement and draft models differed. Why it matters: The release makes DeepSeek’s production inference engine and kernels available for inspection, while the LMSYS results position SGLang as a complete serving alternative with matching or higher measured performance in several configurations. What do you think?
Open the headline: source, AI briefing and more →Comments
…