Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Artificial Analysis Decision Index v2

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 1

  1. Artificial Analysis releases Decision Index v2

    Artificial Analysis released Decision Index v2, whose new Decision Score combines answer correctness with confidence calibration. Private datasets now make up 45% of the index, up from 25% in v1. The update adds the 1,500-document private AA-Paperclip benchmark and the AA-Symmetry consistency test, adds an implementation of the ONET benchmark, removes WildGuardTest because of high saturation, and reduces When2Call’s weighting from 25% to 10%. Claude Haiku 5.5 (Max) and GPT-6 Luna (Max) scored 77 and 76, respectively, while Drex Max v1.1 and GLiDE v1.2 tied at 73 among specialist decision models. GPT-6 Luna cost $1,530 per million tasks and took 11.34 seconds per task, compared with $102 and 275 milliseconds for Drex Max v1.1. GLiDE v1.2 cost $780 per million tasks, nearly eight times Drex Max v1.1’s cost, because it bills the document input for every question.

    ArtificialAnlys (X) · 🔥 0

Experience and discussion from the community

Share my Artificial Analysis Decision Index v2 experienceAsk about Artificial Analysis Decision Index v2

Nobody has shared their experience with Artificial Analysis Decision Index v2 yet.