TRACE Aligns FP4 Training and Rollout for MoE Reinforcement Learning
Alibaba Qwen and Ohio State University researchers proposed TRACE, a framework that uses rollout-side FP4 quantization outcomes to guide training-side rounding for reinforcement learning of MoE language models. Evaluated on four models across reasoning, coding, and long-horizon RL tasks, the authors report performance comparable to BF16 rollout with joint FP4 weights, activations, and KV cache, plus up to a 5.4× rollout speedup. A compact caching scheme stores selected mantissa and scale information to reduce the overhead of transferring rollout guidance.
The paper reports that aligning the training and rollout quantization paths can preserve BF16-like RL performance while accelerating FP4 rollout generation, directly addressing a significant compute bottleneck in MoE model post-training.
Computer Science > Machine Learning
arXiv:2610.07767 (cs)
[Submitted on 6 Oct 2026]
Title:TRACE: Rollout-Guided Quantization-Aware Training for FP4 Reinforcement Learning of MoE Language Models
Authors:Xin Wang, Hao Yu, Zhengyang Zhuge, Bochao Mao, Zheng Li, Junda Feng, Yuyan Luo, Yi Zhang, Yizhong Cao, Mi Zhang, Dayiheng Liu, Jianwei Zhang
View a PDF of the paper titled TRACE: Rollout-Guided Quantization-Aware Training for FP4 Reinforcement Learning of MoE Language Models, by Xin Wang and 11 other authors
#TRACE#Alibaba Qwen#Ohio State University#Qwen3.5-35B-A3B#Qwen3.5-122B-A10B#Qwen3.8-Flash-Next#Qwen3.8-2.4T-A95B