TPPO
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- TPPO: Tracking-Value PPO tops standard PPO for LM reasoning
An arXiv preprint introduces Tracking-Value PPO (TPPO), which analyzes how successive prefix values relate and shows they admit a Bayesian decomposition with additive updates in log-odds space; the method uses a cumulative value parameterization that sums position-wise scalar outputs into value logits, learned from terminal-outcome supervision. The authors report aggregate mathematical reasoning gains over PPO of 7.0 and 8.4 points on Qwen3-4B-Base and Qwen3-8B-Base respectively, along with improved out-of-distribution generalization. They say gains also extend to Qwen3-4B, Qwen2.5-Math-1.5B and Qwen3.5-9B, plus countdown and scientific coding tasks, with code released publicly.
Hugging Face · Papers · 🔥 0
Experience and discussion from the community
Share my TPPO experienceAsk about TPPO
Nobody has shared their experience with TPPO yet.