Agent Arena compares frontier models’ cost and duration
Agent Arena reports that nearly 30,000 tasks over the previous 14 days ran for more than an hour, with the average model making 332 tool calls. For one hour of work, median costs were $10.51 per task with Gemini 4 Carbon, $21.13 with Claude Opus 5.5 and $31.08 with GPT-6 Astra. Among the top models from five labs, task-duration results differed by 55 minutes. Claude Opus 5.5 ranked first in Agent Arena but cost about twice as much per hour as Gemini 4 Carbon.
The findings quantify a trade-off between model capability and operating cost on long-running agent tasks. Agent Arena says the right model choice can reduce costs by as much as 66% in these long-horizon tasks.
Models by @GoogleDeepMind scale better on cost than their rivals.
Gemini 4 Carbon is the prime example. As tasks get more demanding, every rival gets more expensive relative to Carbon.
For a typical task, GPT-6 Astra by @OpenAI costs 3.4× as much, Claude Opus 5.5 by @AnthropicAI costs 3.9× as much, and GPT-6.1 Sol is the cheaper option at about half of Carbon’s cost (0.54×).
Across the 5% most expensive sessions, Carbon’s position improves against all three. Opus edges up to 4.2×, Astra’s cost roughly doubles relative to Carbon, and Sol’s discount shrinks.
This in turn can translate into meaningful savings across real-world tasks such as building an Android security app, one of many long-horizon tasks that our Agent Arena users delegate to their agents.
@arena: As tasks get more complex, @GoogleDeepMind and @XiaomiMiMo are the biggest winners in the Agent Arena. On cheaper tasks, @OpenAI and @AnthropicAI hold 10 of the 11 frontier spots. On the most expensive 5% of sessions, they hold just 7. The freed spots go to Google’s Gemini 4 Argon and Xiaomi’s MiMo-V2.6 Flash and Pro. These shifts matter as sessions costing $100 or more are not uncommon across real-world use cases coming from Agent Arena. Even within the last 2 weeks, more than 1,000 sessions crossed that boundary, and choosing a suboptimal model might have had a meaningful impact on that final bill. https://x.com/arena/status/2118035889175838853
#Agent Arena#Gemini 4 Carbon#Claude Opus 5.5#GPT-6 Astra#GPT-6.1 Sol#Muse Wonder Full#Kimi K3.1 Pro#Gemini 4 Argon#MiMo-V2.6 Flash#MiMo-V2.6 Pro#Google DeepMind#OpenAI