Artificial Analysis Decision Index v2
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- Artificial Analysis releases Decision Index v2
Artificial Analysis released Decision Index v2, whose new Decision Score combines answer correctness with confidence calibration. Private datasets now make up 45% of the index, up from 25% in v1. The update adds the 1,500-document private AA-Paperclip benchmark and the AA-Symmetry consistency test, adds an implementation of the ONET benchmark, removes WildGuardTest because of high saturation, and reduces When2Call’s weighting from 25% to 10%. Claude Haiku 5.5 (Max) and GPT-6 Luna (Max) scored 77 and 76, respectively, while Drex Max v1.1 and GLiDE v1.2 tied at 73 among specialist decision models. GPT-6 Luna cost $1,530 per million tasks and took 11.34 seconds per task, compared with $102 and 275 milliseconds for Drex Max v1.1. GLiDE v1.2 cost $780 per million tasks, nearly eight times Drex Max v1.1’s cost, because it bills the document input for every question.
ArtificialAnlys (X) · 🔥 0
Experience and discussion from the community
Share my Artificial Analysis Decision Index v2 experienceAsk about Artificial Analysis Decision Index v2
Nobody has shared their experience with Artificial Analysis Decision Index v2 yet.