Terminal-Bench 2.0
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 2
- Stanford study finds Terminal-Bench 2.0 defects alter agent scores
A Stanford study summarized by 掘金 found that 28 of 89 tasks in Terminal-Bench 2.0 needed fixes, including 14 newly identified defects, and the repairs shipped in Terminal-Bench 2.1. Re-evaluating 19 agents produced aggregate accuracy more than 10 percentage points higher after the fixes, with some agents moving three ranking positions. The study also found that timeouts accounted for more than half of failed trials under the default budget, while extending the budget nearly erased a 20-point gap between harnesses.
掘金 · 人工智能 · 🔥 0 - ReTerm trains terminal agents to recover from their own failed tasks
An arXiv paper submitted on 13 October presents ReTerm, a self-improving terminal agent trained by learning how to recover from its own failed terminal tasks. The authors built a data pipeline producing ReTerm-10K, a dataset of recovery demonstrations for supervised fine-tuning cold start, and propose ReTerm-GRPO, an online RL algorithm that forks from the agent's own failure states to generate multiple independent self-recovery attempts and learns from whether they complete the task. Built on Qwen3.5-9B, ReTerm-9B reaches 34.8% Pass@1 on Terminal-Bench 2.0, which the authors say is 14.6 points above the base model, and they report consistently higher recovery rates across error categories than standard GRPO training; code, data and model are stated to be released.
Hugging Face · Papers · 🔥 0
Experience and discussion from the community
Share my Terminal-Bench 2.0 experienceAsk about Terminal-Bench 2.0
Nobody has shared their experience with Terminal-Bench 2.0 yet.