Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Open-Source Models

DeepSeek Opensources DeepInfer; LMSYS Benchmarks SGLang Against It

DeepSeek has open-sourced DeepInfer, the inference engine behind its service, together with kernels used by the system. LMSYS tested SGLang against DeepInfer using DeepInfer’s benchmarks on the same hardware, reporting up to 1.04x prefill throughput on four NVIDIA GB300 or four B300 GPUs and decode steps up to 7.4% shorter on 64 GB300 GPUs. The comparison is not fully equivalent because request-token patterns, averaging methods, Engram table placement and draft models differed. Why it matters: The release makes DeepSeek’s production inference engine and kernels available for inspection, while the LMSYS results position SGLang as a complete serving alternative with matching or higher measured performance in several configurations. What do you think?

Open the headline: source, AI briefing and more →

Comments

…