vLLM首日支持Kimi K3.1 Pro
vLLM宣布首日支持Moonshot AI的Kimi K3.1 Pro,覆盖NVIDIA和AMD GPU、Google TPU以及Alibaba T-Head Zhenwu PPU。实现包括稀疏注意力支持,以及统一存放MLA潜变量、索引器键和KDA状态的KV缓存,并配有专用解码与预填充路径。vLLM还报告了长上下文、吞吐和强化学习基准提升,但现有报告未提供独立验证。
此次更新让部署团队可立即在多种加速器平台上运行Kimi K3.1 Pro,新缓存和稀疏注意力路径则面向高效的长上下文推理。
Kimi K3.1 Pro is Moonshot AI's most capable open-weight model: a 2.8T-parameter (104B active) multimodal MoE with a 1M-token context that matches the strongest closed-source models on coding, agentic and knowledge benchmarks.
The main change is attention: K3.1 Pro adds Sparse Attention (DSA) to all 24 full-attention layers for efficiency. vLLM supports it on day 0 on NVIDIA GPUs, AMD GPUs, Google TPUs and Alibaba T-Head Zhenwu PPUs.
#vLLM#Kimi K3.1 Pro#Moonshot AI#NVIDIA#AMD#Google TPU#Alibaba T-Head#Zhenwu PPU#FlashInfer#FlashMLA