An arXiv paper introduces DeltaTTG, which learns a reusable progress signal through a shared progress meter for reinforcement learning of multi-turn agents; its supervision target, Token-to-Go (TTG), is the number of tokens remaining to be generated until task completion, used as an observable proxy for remaining completion effort. The method uses low-cost supervision from recorded trajectories of diverse policies without task-specific progress rules, and during actor training, changes in the frozen meter's estimates redistribute outcome-derived advantages across steps without altering the reward. On ALFWorld, PaperSearchQA and BFCLv3, DeltaTTG improves the average score over GRPO by 5.7 and 7.2 points with Qwen3-4B and Llama-3.1-8B-Instruct respectively, outperforms all compared RL baselines on average while keeping actor-training GPU-hours close to GRPO, and the authors also report improved sample efficiency and out-of-domain performance; code is said to be available.

DeltaTTG·Token-to-Go·Qwen3-4B·+5