Penn State study measures coding agents’ termination lag
A Penn State paper titled “Termination Lag: Coding Agents Solve Early but Stop Late” measures the interval between a coding agent’s first correct patch and when it stops. The study logged repository snapshots during agent runs and replayed them against official SWE-bench tests. Across five models, two agent frameworks and three SWE-bench task sets, the report says a median 35%–60% of steps in successful runs occurred after the first correct patch, while some later changes turned a correct solution into an incorrect one. The findings are based here on a secondary account of the paper.
The study treats knowing when to stop as a distinct dimension of coding-agent performance. Continuing after a solution is already correct adds steps and may increase token, latency and compute costs, while later edits can introduce regressions.
💡 一句话总结:Penn State 团队给 coding agent 做了一次「逐步验尸」:每一步都偷拍仓库快照、事后用官方测试回放——结果发现 agent 大多早就把活干完了,中位 35–60% 的步骤花在「已经解对」之后,还有一小撮运行临走前把对的改错。做 agent 产品和天天用 agent 的都该看看。
导语:你肯定见过这个场面
先问一个你可能刚经历过的问题:让 Claude Code、Cursor 或者随便哪个 coding agent 修一个 bug,它吭哧吭哧读了一堆文件,改了一版代码,测试过了,然后呢?
然后它开始「庆祝胜利」:再跑一遍测试、把改动 diff 又看了一遍、翻了几个根本不用碰的文件、写了个额外的检查脚本……最后慢悠悠地跟你说「任务完成了」。你盯着屏幕右下角的 token 计数,心里默算这一波又烧了多少额度。
更气人的剧本是:它中途明明已经改对了,后面又手痒改了两下,把对的改错了,最后提交一个编译不过的补丁。你只好说「再试一次」,它从头再来一遍。听起来离谱?数据就是这么写的。
#Penn State#Termination Lag: Coding Agents Solve Early but Stop Late#OpenHands#mini-SWE-agent#SWE-bench#SWE-bench Verified#SWE-bench Multilingual#SWE-bench Pro#Qwen3-Coder-30B#Laguna-XS.2#Seed-OSS-36B#Claude Haiku 5.5