Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

DeepSeek-V4.1-Flash Serving Optimized on H20-3e

AI summary

A blog reported that tuning prefill and decode parallelism, using FP8 DeepGEMM scoring, improving CUDA graphs and optimizing Indexer kernels reduced mean end-to-end latency by 65.5% and produced 2.91 times the output throughput of a TP4 baseline in the authors’ H20-3e tests. Mean time to first token fell from 12.04 seconds to 1.89 seconds, while mean time per output token after the first rose from 3.20 to 5.62 milliseconds. A four-GPU colocated configuration reduced mean latency by 57.26% and increased throughput by 130.79% against its BF16 TP4 baseline. The report includes a public SGLang branch, a specific commit and environment instructions for reproduction.

On this page

#DeepSeek-V4.1-Flash#NVIDIA H20-3e#SGLang#DSpark

Original · 13 min readRead the original
Group chat

No comments yet. Start the conversation.