Inferact

Headlines · 4

1. 3d
   vLLM adds day-0 support for Reflection's 501B Beam model with Blackwell optimizations

   vLLM published day-0 serving support for Reflection's Beam, a 501B-parameter mixture-of-experts model with 23B active parameters per token, optimized for NVIDIA Blackwell. The engine handles reasoning, tool calling and structured outputs out of the box and can run Beam with tensor, data or expert parallelism, with a Docker image and launch command provided. The post says coding agents autonomously tuned the implementation so batch-1 decode is 2.71× faster and prefill 1.87× faster than the first version, and that Beam's DFlash2 drafter reaches 708 tokens/s for a single request on TP8 with an acceptance length of 3.7. It also describes day-0 reinforcement-learning support built with the SkyRL team, reporting AIME25 average accuracy rising from about 61% to about 79% after 50 training steps.

   vLLM Blog · 🔥 0
2. 4d
   vLLM adds day-0 support for Meta's Muse Spark Open, an 816B open-weight MoE

   vLLM said it worked with Meta Superintelligence Labs to bring day-0 inference support to Muse Spark Open, an open-weight sparse mixture-of-experts model built for long-horizon agentic coding workflows. The model has 816B total parameters with 42B active per token and a 1M-token context window, plus native image, video and audio perception. Meta released weights in BF16 and NVFP4, AMD published MXFP4 weights, and vLLM supports tensor and pipeline parallelism on Nvidia and AMD with initial TPU support. vLLM also shipped a DSpark speculative-decoding draft model and day-0 LoRA reinforcement-learning support. vLLM called it "the most capable US open-weights model to date," a claim not backed by independent benchmarks in the post.

   vLLM Blog · 🔥 0
3. 5d
   vLLM Adaptive Verification works without a confidence head via online estimator

   A vLLM blog post says Adaptive Verification previously required a per-token acceptance estimate, so it only supported DSpark drafters that ship with a confidence head, leaving EAGLE, MTP, DFlash and some DSpark checkpoints unable to use the optimization. The new approach, when no confidence head exists, derives a feature from the log-odds of the drafter's most likely token and fits a lightweight logistic regression online, with one shared slope and one intercept per draft position, continuously updated from the target model's accept/reject decisions. The feature landed in PR #52228 and is enabled with enable_adaptive_verification for draft models without a confidence head, with results reported for DeepSeek-V4-Flash + DSpark and MiMo-V2.5-Pro-FP4 + DFlash.

   vLLM Blog · 🔥 0
4. Oct 9
   vLLM adds day-0 support for NVIDIA Vera Rubin NVL72, claims 7.8x throughput over GB200

   vLLM says it now runs on NVIDIA's Vera Rubin NVL72, with nightly CUDA 13.4 and PyTorch 2.15 container images and support for models from DeepSeek, Moonshot AI's Kimi, Z.ai's GLM and MiniMax. Working with NVIDIA, Red Hat and Inferact, the project reports early results of 7.8x throughput per GPU versus GB200 NVL72 on AgentX at matched interactivity and up to 3.7x higher VLM throughput versus GB300 NVL72 in MLPerf; these are its own early figures. MiniMax separately said its M3 model is now running on Rubin with vLLM, crediting the same collaborators.

   vLLM Blog · 🔥 0

Experience and discussion from the community

Share my Inferact experienceAsk about Inferact