vLLM 弹性 EP 扩缩容停机降至 1 秒级
vLLM 发布技术博客，介绍如何通过异步准备与 NIXL EP 后端，让运行中的 Elastic Expert Parallelism 部署在几乎不中断服务的情况下增减 GPU。在其 DeepSeek V4 Pro 的 GB200 基准测试中，从 16 张 GPU 扩到 20 张时，早先的 Elastic EP 方案会停止服务 100 秒，整体重启则需 197 秒，而新流程把停机降到 0.82 秒；缩回 16 张时从 56 秒降到 1.96 秒。该流程分为后台准备（初始化新 worker、建立目标通信组、通过 NVLink 或 RDMA 传输权重、预热 kernel 与 CUDA 图）和短暂暂停调度器的提交阶段；vLLM 称相关改动已包含在最新 vLLM 与 NIXL 版本中。
文章指出，此前大规模 MoE 服务调整 GPU 数量要么需要重启、要么长时间停止服务，这段时间昂贵的 GPU 被占用却不产生 token；新方案把这段暂停压缩到秒级。
Traditionally, changing the number of GPUs in a vLLM service required adding or removing entire model replicas or restarting the engine with a different GPU count. As models grow larger, each replica needs more GPUs, so scaling by replica adds or removes GPUs in increasingly large increments. Restarting vLLM to scale DeepSeek V4 Pro from 16 to 20 GB200 GPUs stopped serving for 197 seconds (see the performance section for details), leaving those expensive GPUs allocated without generating tokens.
In our first Elastic EP post, we showed how vLLM could add or remove GPUs from a running DEP deployment (data-parallel attention with expert-parallel MoE) without restarting it. While we avoided a restart, serving still stopped throughout reconfiguration as we initialized new workers, prepared the expanded communication groups, loaded model weights, and repeated kernel warmup and CUDA graph capture. In our DeepSeek V4 Pro benchmark, this pause lasted 100 seconds when scaling from 16 to 20 GPUs and 56 seconds when scaling back to 16 GPUs (full details in the performance section).
In this post, we describe how asynchronous preparation and NIXL EP enabled Elastic EP reconfiguration with near-zero downtime. In the same benchmark, we reduced downtime to 0.82 seconds when scaling from 16 to 20 GPUs and 1.96 seconds when scaling back to 16 GPUs, allowing us to scale up and down with almost no impact on serving performance.
#vLLM#NIXL#DeepSeek V4 Pro#NVIDIA GB200#Dynamo#llm-d#Red Hat