GDPO

Headlines · 1

1. 2d
   MRTPO puts each reward on its relevant tokens in multi-reward RL

   A preprint submitted to arXiv (arXiv:2610.21439, 15 Oct 2026) proposes MRTPO, a method for credit assignment in multi-reward reinforcement learning. The authors note that reward-decoupled normalization (GDPO) normalizes each reward separately but still spreads every reward's advantage evenly over the whole response, even though each verifier checks only a few positions. MRTPO instead replays the policy's own response with and without a given reward's verifier information and uses the per-token Jensen-Shannon distance between the two next-token distributions to measure each token's relevance, reassigning that reward's advantage accordingly, without auxiliary models or token-level annotations. The authors report faster convergence and better results than the compared methods on text-to-SQL, multi-constraint instruction following and tool calling; the paper is 15 pages with 8 figures.

   Hugging Face · Papers · 🔥 0

Experience and discussion from the community

Share my GDPO experienceAsk about GDPO