ProAS改进多轮智能体强化学习信用分配
ProAS论文提出一种无需评论器的框架，用于多轮LLM智能体强化学习中的细粒度信用分配。该方法利用动态更新的Agentic Reference Graph，并结合不合理激励屏蔽和非对称惩罚再分配，避免将轨迹级优势均匀分配给所有动作。作者报告，Qwen2.5-1.5B-Instruct和Qwen2.5-7B-Instruct在ALFWorld上的最高成功率分别为91.9%和94.3%，在WebShop上分别为66.7%和70.8%，训练收敛速度最高提升2.44倍。
ProAS提供了一种无需引入评论器即可改善多轮智能体训练中动作级信用分配的方法，论文同时报告了更高的基准成绩和更快的收敛速度。
Computer Science > Machine Learning
arXiv:2610.18683 (cs)
[Submitted on 13 Oct 2026 (v1), last revised 25 Oct 2026 (this version, v2)]
Title:ProAS: A Progress-Aware Advantage Shaping Framework for Multi-Turn Agentic Reinforcement Learning
Authors:Bowen Ding, Mingda Wan, Yuhan Chen, Jiyao Yuan, Qi Zhu, Futing Wang, Heyuan Deng, Fei Mi, Lifeng Shang, Tao Lin
View a PDF of the paper titled ProAS: A Progress-Aware Advantage Shaping Framework for Multi-Turn Agentic Reinforcement Learning, by Bowen Ding and 9 other authors
#ProAS#Agentic Reference Graph#Qwen2.5-1.5B-Instruct#Qwen2.5-7B-Instruct#ALFWorld#WebShop