Artificial Analysis overhauls Intelligence Index with private evaluations
Artificial Analysis launched Intelligence Index v5, with 60% of its evaluations now based on private test sets or held-out answers. The update adds FrontierCode 2.0 and Terminal-Bench-Science, upgrades three existing benchmarks, and introduces Progress Index v1 to track model capability over time; Claude Opus 5.5 leads the initial v5 ranking with 58 points.
The overhaul shifts leading model rankings toward more realistic coding and scientific tasks while making a majority of the composite score impossible to reproduce independently.
RT Artificial Analysis Announcing Artificial Analysis Intelligence Index v5, featuring 2 new benchmarks and 3 upgrades. For the first time, the majority of the Index is built on private datasets The Artificial Analysis Intelligence Index is our composite measure of model capabilities across tasks reflecting real-world usage today. v5 is our biggest upgrade since v4 launched in January. With v5, 60% of the Index is now based on evaluations with private test sets or held-out answers. Alongside Intelligence Index v5, we’re launching the Artificial Analysis Progress Index v1, which tracks model capability over time on one continuous scale. New to the Intelligence Index: ➤ FrontierCode 2.0: Cognition's newly-launched private benchmark of real-world coding tasks, graded on whether a repository maintainer would merge the agent's change ➤ Terminal-Bench-Science: 70 agentic scientific research tasks, run in a terminal environment Upgraded: ➤ HLE-Diamond (replacing Humanity's Last Exam): a refined 1,000-question version of Humanity's Last Exam, split evenly between reasoning and knowledge questions ➤ AA-Omniscience v1.1: An updated dataset with saturated and public set questions removed, new questions added, and ambiguous or erroneous questions revised ➤ CritPt v2: a private, 40-question hard subset of CritPt v2, revised by the CritPt authors and run with a code-execution tool Staying in the Index: AA-Briefcase, AutomationBench-AA, GDP.pdf and Terminal-Bench 4.0 Removed from the Index: GDPval-AA v2.1, SciCode, and AA-LCR v1.1 Key results: ➤ Claude Opus 5.5 (max) leads at 58, ahead of GPT-6 Astra (max, 57) and GPT-6.1 Sol (max, 55): Anthropic and OpenAI hold the top five models.
#Artificial Analysis#Artificial Analysis Intelligence Index v5#Artificial Analysis Progress Index v1#FrontierCode 2.0#Terminal-Bench-Science#Cognition#Claude Opus 5.5#GPT-6 Astra#GPT-6.1 Sol#Kimi K3.1 Pro#DeepSeek V4.1 Flash 1017#GLM-5.3