vLLM v0.31.0 expands serving, speculation, and model support
AI summary
vLLM v0.31.0 adds broad serving and model-support updates, including DeepSeek-V4.1-Flash optimizations, Model Runner V2 speculative decoding, larger-scale expert parallelism, and improved multimodal support. It also introduces the `vllm preload` weight-cache daemon for faster engine restarts and experimental CRIU-based initialized-engine snapshots, alongside several security and breaking changes.
In-depth insight
The release improves the infrastructure available for deploying newer sparse, multimodal, and mixture-of-experts models at scale.