A vLLM blog post details the engineering work behind serving Kimi K3 on AMD Instinct MI355X. It reports that on the AgentX benchmark at 50 tokens/s/user P90 interactivity, an eight-GPU MI355X system delivers roughly 7,567 total tokens/s per GPU versus 6,083 for NVIDIA HGX B300, a 24.4% throughput advantage, and 87.4% more tokens per dollar under SemiAnalysis's modeled TCO of $1.50 versus $2.26 per GPU-hour. The optimizations described include DCP8 partitioning of MLA history across GPUs, removing duplicated MoE output-projection work, fusing the single-token KDA decode path, and using AMD AITER assembly for MLA long-context decoding, which cut time per output token from 43.8 ms to 23.6 ms.

Kimi K3·AMD Instinct MI355X·vLLM·+5