arXiv preprint: long-horizon RL training recipe leans on the Muon optimizer
An arXiv paper submitted on 14 October 2026, "Long-Horizon RL Needs Muon," by Kai Ruan, Jinghao Lin, Tao Chen and three co-authors, examines reinforcement learning for long-horizon agents that must coordinate reasoning, tool use and environment feedback across extended interactions. The authors describe a single-rollout asynchronous recipe that combines the Muon optimizer, token-averaged loss, truncated importance sampling, and auxiliary length and tool-error penalties, and they study optimizer choices and recipe components for repository-level issue resolution across two model families. They report higher training success with the full recipe than with baselines, though the 7-page work-in-progress paper presents these results without independent replication.
The paper tests optimizer choice alongside asynchronous rollouts and loss-weighting choices within one long-horizon agent RL recipe, offering practitioners a concrete, checkable combination, although its gains are so far only author-reported.
Computer Science > Machine Learning
arXiv:2610.20256 (cs)
[Submitted on 14 Oct 2026]
Title:Long-Horizon RL Needs Muon
Authors:Kai Ruan, Jinghao Lin, Tao Chen, Ziqi Zhou, Qianshan Wei, Zihe Huang
View a PDF of the paper titled Long-Horizon RL Needs Muon, by Kai Ruan and 5 other authors