Reddit re-run reports higher dual-NTA rates for newer Claude models
A Reddit poster says a friend re-ran a Stanford AITA benchmark across 27 models, rewriting posts that Reddit judged “not the asshole” from the other participant’s perspective. The reported dual-NTA rate was 35% for Claude Sonnet 5, 30% for Claude Sonnet 5.5, and 14% for the earlier Claude Opus 4.1, while Grok 4.20 scored 73% and Gemini 4 Argon 7%. The poster’s claim that Claude is now the worst among its peers is stronger than the figures shown, and the results remain secondhand until independently reviewed.
The reported dual-NTA rates offer a concrete way to compare how strongly models align with a user’s framing, making the replication relevant to evaluation of sycophancy and model behavior.
One of my friends re-ran a Stanford study on 27 models. In the study, they ask AI, "Am I the asshole?" from both sides of the same fight. I found the read interesting and wanted to share it with you too.
So, they take an AITA post where Reddit said NTA, rewrite it from the other person's side, and ask the model about both. If it says NTA to both people, it's just agreeing with whoever is asking.
Opus 4.1 only did this 14% of the time, the best of any model back then. Most other labs have improved a lot since then, but Claude went the other way. Sonnet 5 is at 35%, and Sonnet 5.5 is the highest of the newest models with 30%. I always thought of Claude as the one that pushes back; now I'll probably discuss my relationships with it.
Grok 4.20 was the worst of all 27 at 73%, worse than GPT-3.5 (from 2023). Grok 4.7 came out six months later, and it's at 20%. I will attribute this speed to testing it on Elon Musk, but it's just my speculation.
Finally, Gemini 4 Argon has the lowest score (7%), but partly because it refused to pick a side on 31 of the stories. Every refusal was on the other person's version, never the original poster's
One example from the posts: a 4-year-old won't hug his dad goodnight, so the dad sits on the floor with his head in his hands, making sad noises. All 27 models said the mom was right to call it out. 8 of them also told the dad he was NTA
You can find the full results; you can replicate them yourself here: heldoutlabs.com/ai-am-i-the-asshole-test
submitted by /u/ahmetfirat [link] [comments]
#Claude Opus 4.1#Claude Sonnet 5#Claude Sonnet 5.5#Grok 4.20#Grok 4.7#Gemini 4 Argon#GPT-3.5#Stanford University