Paper finds weak LLM actors can still be effective judges and teachers
A new arXiv paper, “Actor, Judge, or Teacher? Understanding Decision Models in Long-Horizon Agents,” examines whether language models with poor acting performance can still evaluate actions or teach long-horizon agents effectively. The authors report that judges examining full trajectories could distinguish useful steps, while the JEV teacher raised a Qwen2.5-1.5B policy trained with GRPO from 72.8% to 82.9% success on ALFWorld and from 56.8% to 71.4% on WebShop. These results are author-reported and have not been independently validated.
The paper offers evidence that a model’s value in a long-horizon agent system may extend beyond directly choosing actions, with reported gains in two reinforcement-learning benchmarks.
Computer Science > Artificial Intelligence
arXiv:2611.17167 (cs)
[Submitted on 12 Nov 2026]
Title:Actor, Judge, or Teacher? Understanding Decision Models in Long-Horizon Agents
Authors:Haochen Liu, Qianhao Zhu, Zixuan Wang, Dingming Li, Tongbo Chen, Peng Qiu, Yingqi Qu, Weiming Lu, Jun Xiao, Yongliang Shen
View a PDF of the paper titled Actor, Judge, or Teacher? Understanding Decision Models in Long-Horizon Agents, by Haochen Liu and 9 other authors