Qwen3.8-2.4T-A95B

Headlines · 1

1. Oct 5
   TRACE Aligns FP4 Training and Rollout for MoE Reinforcement Learning

   Alibaba Qwen and Ohio State University researchers proposed TRACE, a framework that uses rollout-side FP4 quantization outcomes to guide training-side rounding for reinforcement learning of MoE language models. Evaluated on four models across reasoning, coding, and long-horizon RL tasks, the authors report performance comparable to BF16 rollout with joint FP4 weights, activations, and KV cache, plus up to a 5.4× rollout speedup. A compact caching scheme stores selected mantissa and scale information to reduce the overhead of transferring rollout guidance.

   Hugging Face · Papers · 🔥 0

Experience and discussion from the community

Share my Qwen3.8-2.4T-A95B experienceAsk about Qwen3.8-2.4T-A95B