Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

AA-PersonalityBench compares open-weight and proprietary model behavior

AI summary

Artificial Analysis compared the 10 open-weight and 10 proprietary models with the highest scores in its Intelligence Index using AA-PersonalityBench. It percentile-ranked each model from 1 to 100 across 30 personality traits and found that the leading open-weight models averaged 32 points below leading proprietary models on truthfulness. Open-weight models also ranked lower on pragmatism and structure, but higher on slop and bottom-line summarizing.

In-depth insight

The benchmark highlights systematic behavioral differences between leading open-weight and proprietary models beyond their Intelligence Index rankings.

In AA-PersonalityBench, the top open weights models on average rank lower than the top proprietary models on truthfulness, pragmatism and structure, and higher on slop and summarizing the bottom line

We compared the 10 open weights and 10 proprietary models measured with the highest Artificial Analysis Intelligence Index scores. Each model is percentile ranked from 1 to 100 on each of 30 personality traits. On average, the top open weights models rank 32 points lower than the top proprietary models on Truthful, the largest gap of any trait.

Average rank, top proprietary vs. top open weights:

➤ Open weights rank lower: Truthful (79 vs. 47), Pragmatic (79 vs. 49), Natural Sounding (77 vs. 52), Evidence-Based (76 vs. 47), Structured (73 vs. 45) and Literate (70 vs. 47)

➤ Open weights rank higher: Slop (30 vs. 49), Bottom-Line Summary (47 vs. 62) and Scannable (41 vs. 53)

#Artificial Analysis#AA-PersonalityBench#Artificial Analysis Intelligence Index

Original · 1 min readRead the original
Group chat

No comments yet. Start the conversation.