SGLang and Miles tuned for NVIDIA Vera Rubin, serving Kimi K3 NVFP4
The SGLang & Miles team says it received early access to two NVIDIA Vera Rubin nodes (8 GPUs) and brought both SGLang inference and Miles RL training up on the platform. For serving the Kimi K3 NVFP4 checkpoint, it retuned attention, MoE/communication and KDA kernels, reporting a 16% faster deeper MLA pipeline at batch 16 and 128k context, a 20% faster split-KV reduction, a 5.9% end-to-end gain from fusing the MoE tail, and 276 fewer kernel launches per decode step. On the training side, Miles uses SGLang for rollout and Megatron for training, and ran end-to-end RL out of the box on a single 4-GPU tray with Qwen3-30B-A3B and DeepSeek-V4-Flash, plus agentic RL with 64 sandboxes on the Arm-based Vera CPU on SWE-bench Verified. The team also cites Cognition as reporting up to a 4.8x increase in total token throughput on Vera Rubin NVL72 versus GB200 NVL72 using its own private SGLang fork.
It is an early proof point that open-source inference and RL stacks are coming online for the next GPU generation, offering a first signal on serving throughput and cost for very large models.