AITER

Headlines · 2

1. Oct 30
   vLLM v0.33.0rc2 fixes AITER GEMM issue on ROCm gfx950

   vLLM v0.33.0rc2 includes a ROCm bugfix that skips AITER tuned GEMM when M*N is at least 2^31 on gfx950. The release note says the change was cherry-picked from commit 6d01893.

   vLLM Releases · 🔥 0
2. Oct 16
   vLLM: Kimi K3 on AMD MI355X tops NVIDIA HGX B300 throughput per GPU

   A vLLM blog post details the engineering behind serving Kimi K3 on an eight-GPU AMD Instinct MI355X system with vLLM, covering cache reuse of conversation state, eliminating duplicate MoE work across GPUs, fusing memory traffic, an AMD AITER assembly path that speeds long-context MLA decoding, and parallelized KDA prefill and speculative decoding. Citing a SemiAnalysis InferenceX snapshot from October 9, 2026, it reports about 7,567 total tokens/s/GPU at 50 tokens/s/user P90 interactivity versus 6,083 for NVIDIA HGX B300, a 24.4% advantage. Using modeled TCO of $1.50 versus $2.26 per GPU-hour, it claims 87.4% more tokens per dollar; the figures come from a vendor-adjacent source and modeled cost assumptions, with no independent replication reported.

   vLLM Blog · 🔥 0

Experience and discussion from the community

Share my AITER experienceAsk about AITER