ORL open-sourced: diffusion LLM hits 2,788 tokens/s on a single RTX 5090
A research team posted ORL (On-Policy Residual Learning), a method for making diffusion language models converge in fewer forward passes so they can finalize more tokens per step, and open-sourced the models, training code, evaluation pipeline and a custom inference engine called dlmrt. The authors self-report 2,788 tokens/s for LLaDA-2.1-mini plus ORL in NVFP4 on a single RTX 5090, 7,708 tokens/s for DiffusionGemma 26B MoE on 8× B200, up to 6.9× higher end-to-end throughput on QwenDLM-4B, and a HumanEval pass@1 rise from 82.9% to 89.6%. They caution that all figures are batch-size-1 single-request peaks rather than averages over arbitrary prompts or aggregate throughput, and that the DFlash and EAGLE3 comparisons are end-to-end system tests in which ORL runs on dlmrt while the speculative decoders run on vLLM.
If the authors' self-reported results reproduce, diffusion LLMs could offer a viable alternative to autoregressive decoding for low-latency single-request use, with code and checkpoints released openly.