Hybrid HiSparse

Headlines · 1

1. Oct 23
   vLLM scales GLM-5.3 to 1M-token contexts on 16 H200 GPUs

   vLLM Blog reports a GLM-5.3 serving deployment that uses 16 NVIDIA H200 GPUs, divided into separate prefill and decode nodes. Prefill Context Parallelism splits prompts by token, Decode Context Parallelism shards the KV cache, and Hybrid HiSparse keeps decode KV data in host memory. On AgentX, the tested PCP8 × DCP8 configuration with Hybrid HiSparse reported 1.7–3.2× per-user speed at equal throughput to data-parallel serving and was the only tested configuration that served 1M-token contexts; however, these results came from the project's own contributors.

   vLLM Blog · 🔥 0

Experience and discussion from the community

Share my Hybrid HiSparse experienceAsk about Hybrid HiSparse