Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Artificial Analysis updates Coding Agent Index to v2.1

AI summary

Artificial Analysis said its Coding Agent Index v2.1 replaces FrontierCode v1.1 with FrontierCode-AA 2.0 and updates Terminal-Bench from 4.0 to 4.1. The revised index places Claude Fable 5.2 and Claude Opus 5.5 joint first with scores of 58. Claude Fable 5.2 at max scored 56 as more attempts reached task time limits. Artificial Analysis attributed an average five-point decline in model scores to the FrontierCode-AA 2.0 upgrade. Among configurations scoring at least 48, cost per task ranged from $1.51 to $44.22.

Announcing Artificial Analysis Coding Agent Index v2.1, upgrading FrontierCode and Terminal-Bench. Claude Fable 5.2 and Claude Opus 5.5 share the lead at 58

The Coding Agent Index measures models in agent harnesses; v2.1 of the Index upgrades FrontierCode from 1.1 to 2.0 and Terminal-Bench from 4.0 to 4.1.

Upgraded:

➤ FrontierCode-AA 2.0 (replacing FrontierCode v1.1): our implementation of Cognition's FrontierCode 2.0, comprising 132 real-world coding tasks graded on whether a repository maintainer would merge the agent's change

➤ Terminal-Bench 4.1 (replacing Terminal-Bench 4.0): the same 66 tasks, scored with updated verifiers that improve reliability and resistance to reward hacking

Staying in the Index: SWE-Atlas-QnA-AA

#Artificial Analysis#FrontierCode-AA 2.0#Terminal-Bench 4.1#Claude Fable 5.2#Claude Opus 5.5#Claude Sonnet 5.5#Gemini 4 Carbon#GPT-6.1 Terra#GPT-6.1 Sol#Claude Code#Devin Fusion CLI#Codex

Original · 1 min readRead the original
Group chat

No comments yet. Start the conversation.