vLLM Adds Day-0 Support for Inkling 2 Small
According to vLLM Blog, vLLM now supports Thinking Machines’ Inkling 2 Small, a multimodal mixture-of-experts model with 594B total parameters, 17B active parameters and a context window of up to 1M tokens. The integration supports mixed-precision serving on a single node, LoRA, reinforcement learning through SkyRL and a native DSpark speculator that drafts seven tokens per step. vLLM reports optimizations that reached about 630 tokens per second for a single user at TP4 on GB200, with an acceptance length of 3.5. Its accuracy evaluations on NIAH, MMMU-Pro and HLE reportedly matched runs without speculative decoding. The performance and accuracy figures are the project’s own results in the supplied report.
The integration provides a serving path for Inkling 2 Small with mixed-precision deployment, long-context multimodal inference, speculative decoding and reinforcement-learning and LoRA support.

vLLM supports Thinking Machines' Inkling 2 Small on day 0. It is a multimodal (text and vision) MoE model with 594B total and 17B active parameters and up to 1M tokens of context. The model also comes with a native DSpark speculator. vLLM supports it with optimized performance (up to 630 TPS/user at TP4 and acceptance length 3.5) and also supports LoRA.
TL;DR
- 594B total, 17B active, 1M context. A 48-layer MoE with 384 experts (top-8) that takes images and text and outputs text.
- Architecture upgrade from Inkling. Block attention residuals, attention sink, gated attention, and ViT image encoder.
- Mixed-precision MoE. The MoE uses NVFP4 (layers 0–39) and MXFP8 (layers 40–47), so the model can be served with a single node.
- Native DSpark speculator. DSpark with an RNN head drafts 7 tokens per step.
- Extensive optimizations. Kernel fusion, PDL, highly tuned Gluon attention kernel, and sequence parallelism, achieving up to 630 TPS/user on GB200 at TP4 and acceptance length 3.5.
- RL and LoRA support. vLLM supports RL (via SkyRL) and LoRA for the model.
#vLLM#Thinking Machines#Inkling 2 Small#DSpark#NVIDIA GB200#SkyRL#LoRA