Artificial Analysis plans API release for improved CritPt v2
Artificial Analysis said it worked with the CritPt authors to finalize the CritPt v2 grading server and planned to make it available through its API the following day. The authors improved the robustness of the grading-server logic, and Artificial Analysis incorporated those changes in version 5.0.1 of its Intelligence Index. It then regraded every model, finding small score increases for most models while frontier-model rankings remained unchanged, with scores moving by at most one point.
The update makes the CritPt v2 grading methodology more accessible while showing that its robustness improvements had little effect on frontier-model rankings.
We’ve worked with the CritPt authors to finalize the CritPt v2 grading server and will be making this available to the community via Artificial Analysis’ API tomorrow. As part of this process the authors made robustness improvements to the grading server logic, which we have incorporated as v5.0.1 of the Artificial Analysis Intelligence Index
We regraded every model and observe small score increases for most models. Intelligence Index rankings are unchanged among frontier models, with scores moving by at most 1 point.
Thanks to @MiniHui_zhu, @MinyangTian1 and the rest of the CritPt team for their ongoing collaboration
#Artificial Analysis#CritPt v2#Artificial Analysis Intelligence Index