GDPval

Headlines · 1

1. Oct 24
   Artificial Analysis changes make benchmark scores harder to compare

   Artificial Analysis revised its benchmark methodology several times from v4.1 through v4.3.2, changing test weights, replacing or adding evaluations, retiring GPQA Diamond, and recalibrating GDPval Elo scores, according to its official changelog and methodology. Huxiu AI notes that Kimi K3 fell from 57 at its July evaluation to 44 on the current model page, while Kimi K3.1 Pro received 55 and was expected to lead the open-model ranking once weights were equalized. Because the service did not publish a crosswalk between old and new scores, results from different versions or capture dates should not be compared directly.

   虎嗅 AI · 🔥 0

Experience and discussion from the community

Share my GDPval experienceAsk about GDPval