Terminal-Bench 2
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- HarnessPro recursively evolves agent harnesses with Pareto cost-performance tuning
The paper introduces HarnessPro, a label-free method that evolves AI agent harnesses through recursive self-improvement, aiming to improve task performance while reducing execution cost. Candidate harnesses are evaluated using LLM-generated tests without access to ground-truth solutions or official tests; specialized agents diagnose execution trajectories, plan updates with different performance-cost priorities and implement them, and normalized weighted squared-distance selection yields cost-focused (lite), balanced (medium) and performance-focused (heavy) harnesses. The authors report that on Terminal-Bench 2, SWE-Bench Pro and ProgramBench, HarnessPro's heavy mode with GPT-5.5 and Codex achieved an average pass-rate gain of 2.0 percentage points and a 26.1% average per-task cost reduction relative to RHO, and say the advantage holds with GLM-5.2 and Claude Code; code is available and the 41-page paper is under review.
Hugging Face · Papers · 🔥 0
Experience and discussion from the community
Share my Terminal-Bench 2 experienceAsk about Terminal-Bench 2
Nobody has shared their experience with Terminal-Bench 2 yet.