Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Scale AI's HSS benchmark tests AI 'sixth sense'; top model hits 53.6% vs humans' 93.1%

AI summary

Scale AI and Elorian released Humanity's Sixth Sense (HSS), a benchmark that tests whether models can infer what is not explicitly stated in an image or clip — whether another car fits in a gap, why a woman chasing a bus suddenly slows down, or what number belongs on a blurred beaker label. The set holds 522 open-ended tasks drawn from 288 images and 234 videos totaling 17.6 hours, split into temporal/causal dynamics, physical and spatial reasoning, social understanding, and abstract/contextual inference. Twenty human participants averaged 93.1%, while GPT-6 Astra topped the model leaderboard at 53.6% even at maximum reasoning effort, followed by GPT-6.1 Sol (46.6%), Claude Opus 5.5 (44.6%), Gemini 3.8 Flash (41.6%) and Claude Fable 5.1 (40.8%); the median across 25 models was 30.9%. Attributing 8,573 failures, the team says 94% stemmed from perception or latent-inference errors rather than flawed logic, and wrapping models in Claude Code or Codex agents improved scores on a 388-task subset but barely dented 2D-occlusion-read-as-3D-alignment mistakes. The benchmark, scores and human baseline come from the releasing organizations and have not been independently reproduced.

新智元 2026-10-17 12:22 北京

新智元报道  

AI能读图表、做竞赛题、识别密密麻麻的文字,甚至用上万枚推理Token,把一道视觉题分析得头头是道。

但它可能看不出:追公交的女人为什么突然慢了下来;两辆车之间还能不能再塞进一辆;会议室里谁最心不在焉;一排烧杯里,最后那个看不清的标签到底是多少。

这些问题,人类往往瞄一眼就有答案。

#Scale AI#Elorian#Humanity's Sixth Sense#GPT-6 Astra#GPT-6.1 Sol#Claude Opus 5.5#Gemini 3.8 Flash#Claude Fable 5.1#Claude Code#Codex

Original · 8 min readRead the original
Group chat

No comments yet. Start the conversation.