vLLM用16张H200优化GLM-5.3百万token部署
vLLM Blog 报道了一套 GLM-5.3 服务部署，使用 16 张 NVIDIA H200 GPU，并划分为独立的预填充与解码节点。Prefill Context Parallelism 按 token 切分提示词，Decode Context Parallelism 分片 KV 缓存，Hybrid HiSparse 则让解码阶段将 KV 数据保留在主机内存中。在 AgentX 上，受测的 PCP8 × DCP8 配置配合 Hybrid HiSparse，在吞吐量与数据并行部署相同时报告了 1.7–3.2 倍的每用户速度，也是测试中唯一可服务 100 万 token 上下文的配置；不过，这些结果由项目贡献者自行测量。
TL;DR: Long agentic contexts push GLM-5.3 against two limits on H200s: KV cache memory and prefill speed. We address both on 16 H200s, split into one prefill node and one decode node:
- Prefill Context Parallelism (PCP) splits each prompt across GPUs by token instead of by head, which suits GLM-5.3's sparse attention far better than tensor parallelism.
- Decode Context Parallelism (DCP) shards the prefill engine's KV cache by token, so a 1M context fits on the prefill node.
- Hybrid HiSparse lets decode keep KV in host memory, so a 1M-token request fits.
- On AgentX, PCP8 × DCP8 prefill with Hybrid HiSparse decode gives 1.7–3.2× the per-user speed of a data-parallel deployment at the same throughput, and it is the only configuration we tested that serves 1M-token contexts.
Serving long contexts in memory-constrained deployments
Agentic workloads have long contexts that grow with every turn, which puts pressure on both GPU memory and prefill. On H200s, GLM-5.3's weights take up most of each GPU's memory, leaving little room for the KV cache of those long contexts. And because every turn adds more tool output and file contents to prefill, under load the prefill engines can become the bottleneck for the whole deployment.
#GLM-5.3#vLLM#NVIDIA H200#AgentX#Prefill Context Parallelism#Decode Context Parallelism#Hybrid HiSparse#Model Runner V2#NIXL