AMD Instinct MI355X

Headlines · 4

1. 1d
   vLLM: Kimi K3 on AMD Instinct MI355X tops NVIDIA HGX B300 on AgentX

   A vLLM blog post details the engineering work behind serving Kimi K3 on AMD Instinct MI355X. It reports that on the AgentX benchmark at 50 tokens/s/user P90 interactivity, an eight-GPU MI355X system delivers roughly 7,567 total tokens/s per GPU versus 6,083 for NVIDIA HGX B300, a 24.4% throughput advantage, and 87.4% more tokens per dollar under SemiAnalysis's modeled TCO of $1.50 versus $2.26 per GPU-hour. The optimizations described include DCP8 partitioning of MLA history across GPUs, removing duplicated MoE output-projection work, fusing the single-token KDA decode path, and using AMD AITER assembly for MLA long-context decoding, which cut time per output token from 43.8 ms to 23.6 ms.

   vLLM Blog · 🔥 5
2. 1d
   Vinci4D pitches physics-driven AI simulation to speed chip design

   Silicon Valley startup Vinci4D presented its physics-driven AI simulation platform at SEMICON West, saying engineers can run simulations a day after a design change instead of roughly two weeks later, and claiming the software runs up to 1,000 times faster than conventional simulation tools with higher accuracy. The company says its pretrained physics models are trained on physics equations rather than customer data, can analyze multiple physical phenomena simultaneously, and can run on inference platforms such as AMD's MI355X. Vinci4D closed a $250 million Series B round at a $1.5 billion valuation earlier this month, jointly led by Advent, Temasek and Xora, with AMD Ventures also participating.

   SiliconANGLE AI · 🔥 8
3. 1d
   SGLang and Miles add day-0 support for Muse Spark Open

   An LMSYS blog post says SGLang and Miles have added day-0 support for Muse Spark Open, an open-weight multimodal mixture-of-experts model. Its text backbone has 816B parameters with roughly 42B active parameters per token, accepts text, image, video and audio inputs, supports a 1M-token context window, and ships NVFP4 and BF16 checkpoints. The SGLang integration adds sequence parallelism with NVFP4 MegaMoE for prefill, multimodal preprocessing optimizations and DFlash V2 speculative decoding; the team reports that on 8 NVIDIA B300 GPUs with the NVFP4 checkpoint, an OpenHands multi-turn agentic coding replay reaches 14.8K total tokens/sec per GPU at concurrency 1 and 80.3K at concurrency 128.

   LMSYS Blog · 🔥 5
4. 2d
   vLLM ships day-0 support for Meta's Muse Spark Open 816B open-weight MoE model

   vLLM said it worked with Meta Superintelligence Labs to bring day-0 serving support to Muse Spark Open, an open-weight sparse mixture-of-experts model. According to the vLLM blog, the model has 816B total parameters with 42B active per token and a 1M-token context window, ships in BF16 and NVFP4 (AMD released MXFP4 weights), and handles image, video and audio input through the same Perception Encoder used by Muse Glimmer. vLLM also offers a DSpark speculative-decoding draft model trained by Inferact, support on NVIDIA, AMD Instinct MI355X and TPU, and a LoRA reinforcement-learning recipe that lifted AIME-2024 (mean@8) from 0.56 to 0.85 after 150 GRPO steps on 16 GB300 GPUs.

   vLLM Blog · 🔥 0

Experience and discussion from the community

Share my AMD Instinct MI355X experienceAsk about AMD Instinct MI355X