Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Artificial Analysis adds custom index builder to Data Playground

AI summary

Artificial Analysis introduced a custom-index builder in its Data Playground, letting users combine benchmarks, adjust weights and plot the resulting scores against other chart axes. According to the vendor, a model receives a custom-index score only when it has results for every evaluation included. The broader playground also offers formula metrics, launch-date playback, heat maps, grouped box plots and a workbench, and access is included with Artificial Analysis Pro.

In-depth insight

The release gives AI researchers and practitioners a vendor-provided way to define and visualize custom model evaluations, though the capabilities have not been independently verified in the supplied report.

Build your own custom index in the Artificial Analysis Data Playground

Add any Artificial Analysis benchmarks, customize their weights, and plot the results with other chart axis. Models receive a score only if they have results for every evaluation in your index, consistent with the Intelligence Index methodology.

Also in the Data Playground: ➤ Formula metrics: combine any metrics with +, -, *, / and functions such as log and sqrt ➤ Playback: play a scatter chart forward by launch date, with the frontier drawn as it stood at each date ➤ Heat maps across many evaluations, and box plot distributions by group, such as open weights vs proprietary ➤ A workbench with sheets, Columns/Rows shelves and a dashboard view

The Data Playground is available with Artificial Analysis Pro.

#Artificial Analysis#Artificial Analysis Data Playground#Artificial Analysis Pro

Original · 1 min readRead the original
Group chat

No comments yet. Start the conversation.