DeepSeek-V3-Lite

Headlines · 1

1. Oct 21
   SGLang adds elastic expert-parallel scaling for live MoE deployments

   SGLang now supports runtime scale-up for MoE inference deployments, adding GPU ranks in fine-grained increments without replacing the running deployment or restarting it. LMSYS Blog reported an illustrative expansion from 20 to 24 GPUs and said project-run tests on eight H100 GPUs with DeepSeek-V3-Lite FP8 reached the first completed request 5.3 to 8.2 times faster than restarting, though the measurements were not independently corroborated. Automatic rollback is not yet supported after joining ranks enter Mooncake WORLD, while scale-down and recovery after scale-up remain future work.

   LMSYS Blog · 🔥 0

Experience and discussion from the community

Share my DeepSeek-V3-Lite experienceAsk about DeepSeek-V3-Lite