CuTe DSL

Headlines · 1

1. Oct 16
   vLLM Uses Coding Agents to Optimize Kimi K3 Inference on GB300

   The vLLM team said it used agent-assisted development to build a unified CuTe DSL decode path for Kimi K3 on NVIDIA GB300 systems, covering attention, MoE and communication across 8 or 16 GPUs. The team reported more than 840 tokens/s at batch size 1 on 16 GPUs with DSpark, 1.5–2× higher decode throughput in several tested configurations and up to 1.8× higher P90 interactivity on AgentX. SemiAnalysis separately reported 423 tok/s/user on GB300 NVL72, a 1.68× speedup, while the figures depend on different configurations and measurement methods.

   vLLM Blog · 🔥 0

Experience and discussion from the community

Share my CuTe DSL experienceAsk about CuTe DSL