DiffusionGemma 26B MoE

Headlines · 1

1. 3d
   ORL open-sourced: diffusion LLM hits 2,788 tokens/s on a single RTX 5090

   A research team posted ORL (On-Policy Residual Learning), a method for making diffusion language models converge in fewer forward passes so they can finalize more tokens per step, and open-sourced the models, training code, evaluation pipeline and a custom inference engine called dlmrt. The authors self-report 2,788 tokens/s for LLaDA-2.1-mini plus ORL in NVFP4 on a single RTX 5090, 7,708 tokens/s for DiffusionGemma 26B MoE on 8× B200, up to 6.9× higher end-to-end throughput on QwenDLM-4B, and a HumanEval pass@1 rise from 82.9% to 89.6%. They caution that all figures are batch-size-1 single-request peaks rather than averages over arbitrary prompts or aggregate throughput, and that the DFlash and EAGLE3 comparisons are end-to-end system tests in which ORL runs on dlmrt while the speculative decoders run on vLLM.

   r/MachineLearning · 🔥 0

Experience and discussion from the community

Share my DiffusionGemma 26B MoE experienceAsk about DiffusionGemma 26B MoE