PAR trains looped LLM iterations on problem progress, reporting 16x length extrapolation
An arXiv paper submitted on 20 Oct 2026 introduces Progress-Aligned Recurrence (PAR), which trains the prediction at iteration k to match the state after the first k operations and rescales recurrent states to their first-iteration norms. On the swap-S5 state-tracking task, a Qwen2.5-0.5B model trained on at most 32 moves reached 95.5% mean full-state accuracy at 512 moves, or 16x the training length, versus 55.7% reported for RLT and 0.8% for explicit chain of thought on the same backbone. Controlled ablations show that removing norm anchoring drops the extrapolation factor from 16x to below 2.2x, and replacing progress-aligned targets with final-answer supervision reduces it to at most 8x; a learned completion readout decides when to answer without a supplied stopping depth.
The authors conclude that training intermediate predictions and recurrent-state scale together is what makes additional recurrent depth useful for longer problems, rather than simply running more iterations at test time.
Computer Science > Computation and Language
arXiv:2610.27628 (cs)
[Submitted on 20 Oct 2026]
Title:PAR: Progress-Aligned Recurrence for Length Generalization in State Tracking
Authors:Pan Wang, Yihao Hu, Guangyuan Dong, Yitong Lin, Hang Wang, Zimi Liu, Zhihao Wen, Jianshe Li, Jiang-Ming Yang, Wei Wu, Yongqi Tong
View a PDF of the paper titled PAR: Progress-Aligned Recurrence for Length Generalization in State Tracking, by Pan Wang and 10 other authors