vLLM 为 Inkling 2 Small 提供首日支持
据 vLLM Blog 报道,vLLM 现已支持 Thinking Machines 的 Inkling 2 Small。这是一款多模态混合专家模型,拥有 594B 总参数、17B 激活参数和最高 1M token 的上下文窗口。此次集成支持单节点混合精度服务、LoRA、通过 SkyRL 实现的强化学习,以及原生 DSpark 推测器;后者每步草拟 7 个 token。vLLM 称,优化后的实现在 GB200 的 TP4 配置下,单用户解码速度约为每秒 630 token,接受长度为 3.5。其在 NIAH、MMMU-Pro 和 HLE 上的准确率评测结果据称与未启用推测解码时的运行结果一致。性能和准确率数据均为所提供报告中的项目方测试结果。
此次集成为 Inkling 2 Small 提供了混合精度部署、长上下文多模态推理、推测解码以及强化学习和 LoRA 支持的服务路径。

vLLM supports Thinking Machines' Inkling 2 Small on day 0. It is a multimodal (text and vision) MoE model with 594B total and 17B active parameters and up to 1M tokens of context. The model also comes with a native DSpark speculator. vLLM supports it with optimized performance (up to 630 TPS/user at TP4 and acceptance length 3.5) and also supports LoRA.
TL;DR
- 594B total, 17B active, 1M context. A 48-layer MoE with 384 experts (top-8) that takes images and text and outputs text.
- Architecture upgrade from Inkling. Block attention residuals, attention sink, gated attention, and ViT image encoder.
- Mixed-precision MoE. The MoE uses NVFP4 (layers 0–39) and MXFP8 (layers 40–47), so the model can be served with a single node.
- Native DSpark speculator. DSpark with an RNN head drafts 7 tokens per step.
- Extensive optimizations. Kernel fusion, PDL, highly tuned Gluon attention kernel, and sequence parallelism, achieving up to 630 TPS/user on GB200 at TP4 and acceptance length 3.5.
- RL and LoRA support. vLLM supports RL (via SkyRL) and LoRA for the model.
#vLLM#Thinking Machines#Inkling 2 Small#DSpark#NVIDIA GB200#SkyRL#LoRA