Controlled study examines learned unmasking for diffusion language models
A preprint submitted to arXiv on October 22, 2026 presents a controlled study of learned unmasking policies for masked diffusion language models, separating the effects of teacher labels, imitation states, policy inputs, capacity, and compute overhead. Its 856K-parameter trajectory-contraction policy achieved the reported best accuracy–NFE frontier and highest GSM8K throughput on LLaDA-8B-Instruct, with competitive results on MATH-500 and HumanEval and transfer largely to Dream-7B-Instruct.
The preprint reports that a small learned policy can improve the accuracy–compute trade-off in diffusion language models while identifying which supervision and input choices matter.
Computer Science > Artificial Intelligence
arXiv:2610.31576 (cs)
[Submitted on 22 Oct 2026]
Title:What Makes a Good Unmasking Policy? A Controlled Study of Learned Unmasking in Diffusion Language Models
Authors:Jinya Sakurai, Xun Xu, Jaehong Yoon
View a PDF of the paper titled What Makes a Good Unmasking Policy? A Controlled Study of Learned Unmasking in Diffusion Language Models, by Jinya Sakurai and 2 other authors
#LLaDA-8B-Instruct#Dream-7B-Instruct#GSM8K#MATH-500#HumanEval