HiSparse

Headlines · 1

1. Oct 29
   vLLM adds day-0 support for Mistral Large 4

   vLLM announced day-0 support for Mistral Large 4, Mistral’s native multimodal mixture-of-experts model, through the dedicated vllm/vllm-openai:mistral-large-4 Docker image. The implementation supports FP8 and NVFP4 checkpoints, image inputs, built-in DSpark speculative decoding, long-context serving and AMD Instinct MI350 deployment with ROCm. It also includes generic improvements to context parallelism, NIXL-based disaggregated serving and HiSparse host-memory KV offload. Performance and quality figures reported by vLLM were not independently verified in the supplied material.

   vLLM Blog · 🔥 0

Experience and discussion from the community

Share my HiSparse experienceAsk about HiSparse