vLLM adds day-0 support for Kimi K3.1 Pro
vLLM announced day-0 support for Moonshot AI's Kimi K3.1 Pro across NVIDIA and AMD GPUs, Google TPUs, and Alibaba T-Head Zhenwu PPUs. The implementation adds sparse-attention support, a unified KV cache for MLA latent data, indexer keys and KDA state, and specialized decode and prefill paths. vLLM also reported long-context, throughput and reinforcement-learning benchmark gains, but these measurements have not been independently corroborated in the supplied report.
The update gives deployment teams immediate access to Kimi K3.1 Pro on several accelerator platforms, while the new cache and sparse-attention paths target efficient long-context inference.
Kimi K3.1 Pro is Moonshot AI's most capable open-weight model: a 2.8T-parameter (104B active) multimodal MoE with a 1M-token context that matches the strongest closed-source models on coding, agentic and knowledge benchmarks.
The main change is attention: K3.1 Pro adds Sparse Attention (DSA) to all 24 full-attention layers for efficiency. vLLM supports it on day 0 on NVIDIA GPUs, AMD GPUs, Google TPUs and Alibaba T-Head Zhenwu PPUs.
#vLLM#Kimi K3.1 Pro#Moonshot AI#NVIDIA#AMD#Google TPU#Alibaba T-Head#Zhenwu PPU#FlashInfer#FlashMLA