FrontierHarness Eval v1.1
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- FrontierHarness Eval v1.1 compares agent harnesses across five models
FrontierHarness and Baseten published FrontierHarness Eval v1.1, a benchmark covering 22 harness configurations, five models, 62 terminal tasks and four attempts per task. The report says it used 83 billion tokens across 27,280 agent runs, comparing pass rates and list-price inference costs. It found that each model had a different best-value harness under the benchmark’s cost-versus-pass-rate rule: Codex CLI with GLM-5.3, Pi with GLM-5.3-Flash, ZCode with DeepSeek V4.1 Flash 1017, Oh My Pi with Muse and Alma with Kimi-K3. OpenCode v2 had the lowest average cost per task at $0.561, while ZCode had the highest pass rate at 61.9%. The report estimated that failed and timed-out trials accounted for 64.5% of list-price inference costs.
Hacker News 首页(100+ 分) · 🔥 0
Experience and discussion from the community
Share my FrontierHarness Eval v1.1 experienceAsk about FrontierHarness Eval v1.1
Nobody has shared their experience with FrontierHarness Eval v1.1 yet.