DeepInfer kernels
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- SGLang Reports Matching or Beating DeepInfer
DeepSeek open-sourced DeepInfer, the inference engine behind its own service, along with related kernels. In project-run tests using DeepInfer’s benchmarks and the same hardware, SGLang reached up to 1.04x prefill throughput on four GB300 or four B300 GPUs and cut decode step time by as much as 7.4% on 64 GB300 GPUs, according to the SGLang report. A 36氪 report said vLLM also approached or partly exceeded DeepInfer under its disclosed configurations. The comparisons used project-reported results and differing workloads and measurement methods.
LMSYS Blog · 🔥 0
Experience and discussion from the community
Share my DeepInfer kernels experienceAsk about DeepInfer kernels
Nobody has shared their experience with DeepInfer kernels yet.