SGLang adds elastic expert-parallel scaling for live MoE deployments
SGLang now supports runtime scale-up for MoE inference deployments, adding GPU ranks in fine-grained increments without replacing the running deployment or restarting it. LMSYS Blog reported an illustrative expansion from 20 to 24 GPUs and said project-run tests on eight H100 GPUs with DeepSeek-V3-Lite FP8 reached the first completed request 5.3 to 8.2 times faster than restarting, though the measurements were not independently corroborated. Automatic rollback is not yet supported after joining ranks enter Mooncake WORLD, while scale-down and recovery after scale-up remain future work.
Elastic expert-parallel scale-up gives large MoE serving deployments a finer-grained way to add GPU capacity without allocating a full replica or restarting the running service.
On this page