AMD Instinct MI350

Headlines · 2

1. Oct 29
   vLLM adds day-0 support for Mistral Large 4

   vLLM announced day-0 support for Mistral Large 4, Mistral’s native multimodal mixture-of-experts model, through the dedicated vllm/vllm-openai:mistral-large-4 Docker image. The implementation supports FP8 and NVFP4 checkpoints, image inputs, built-in DSpark speculative decoding, long-context serving and AMD Instinct MI350 deployment with ROCm. It also includes generic improvements to context parallelism, NIXL-based disaggregated serving and HiSparse host-memory KV offload. Performance and quality figures reported by vLLM were not independently verified in the supplied material.

   vLLM Blog · 🔥 0
2. Oct 21
   PyTorch adds FlyDSL backend support for AMD MI350 GPUs

   PyTorch Blog reports that AMD Instinct MI350-series GPU users can access FlyDSL kernels through existing PyTorch operator APIs. The backend covers dense and grouped GEMM, MXFP8 and MXFP4 scaled GEMM, attention, RMSNorm and TopK. GEMMs use the optional backend through torch.compile, eligible RMSNorm and TopK calls can dispatch automatically in eager mode, and unsupported inputs retain existing PyTorch implementations. FlyDSL attention kernels are included in AOTriton 0.14.2b or later. Official tests on MI355X reported a 1.10x geometric-mean speedup for the BF16 dense-GEMM suite, while end-to-end vLLM A/B tests reached whole-request speedups of 1.13x for BF16, 2.04x for MXFP8 and 3.01x for MXFP4. These results depend on the workload, model and concurrency, and were not independently verified in the supplied reports.

   PyTorch Blog · 🔥 0

Experience and discussion from the community

Share my AMD Instinct MI350 experienceAsk about AMD Instinct MI350