PPO

Headlines · 2

1. 3d
   Player-1: Learning game control from synthetic experience

   Player-1 is a post-training framework for game-playing agents that aims to learn beyond demonstration data through synthetic experience. The authors first fine-tune computer-use models on 6,044 curated gameplay trajectories and extend the action space with relative pointer motion, sustained inputs and composite actions; they then run multi-environment PPO reinforcement learning in Player-1-Gym, a set of sixteen synthetic 2D and 3D environments with programmatically verifiable rewards and randomized observations and dynamics. The paper reports that Player-1-9B and Player-1-35B-A3B improve MineDojo-300 success over their respective SFT policies by 27.8 and 17.7 percentage points, with Player-1-9B reaching 38.33% versus 12.63% for the strongest evaluated baseline, plus further gains over SFT on ViZDoom-10 and GameWorld. It is an arXiv preprint submitted on 14 October 2026 (arXiv:2610.20584) with 12 authors.

   Hugging Face · Papers · 🔥 0
2. 5d
   TPPO: Tracking-Value PPO tops standard PPO for LM reasoning

   An arXiv preprint introduces Tracking-Value PPO (TPPO), which analyzes how successive prefix values relate and shows they admit a Bayesian decomposition with additive updates in log-odds space; the method uses a cumulative value parameterization that sums position-wise scalar outputs into value logits, learned from terminal-outcome supervision. The authors report aggregate mathematical reasoning gains over PPO of 7.0 and 8.4 points on Qwen3-4B-Base and Qwen3-8B-Base respectively, along with improved out-of-distribution generalization. They say gains also extend to Qwen3-4B, Qwen2.5-Math-1.5B and Qwen3.5-9B, plus countdown and scientific coding tasks, with code released publicly.

   Hugging Face · Papers · 🔥 0

Experience and discussion from the community

Share my PPO experienceAsk about PPO