Humanity's Sixth Sense

Headlines · 1

1. 2d
   Scale AI's HSS benchmark tests AI 'sixth sense'; top model hits 53.6% vs humans' 93.1%

   Scale AI and Elorian released Humanity's Sixth Sense (HSS), a benchmark that tests whether models can infer what is not explicitly stated in an image or clip — whether another car fits in a gap, why a woman chasing a bus suddenly slows down, or what number belongs on a blurred beaker label. The set holds 522 open-ended tasks drawn from 288 images and 234 videos totaling 17.6 hours, split into temporal/causal dynamics, physical and spatial reasoning, social understanding, and abstract/contextual inference. Twenty human participants averaged 93.1%, while GPT-6 Astra topped the model leaderboard at 53.6% even at maximum reasoning effort, followed by GPT-6.1 Sol (46.6%), Claude Opus 5.5 (44.6%), Gemini 3.8 Flash (41.6%) and Claude Fable 5.1 (40.8%); the median across 25 models was 30.9%. Attributing 8,573 failures, the team says 94% stemmed from perception or latent-inference errors rather than flawed logic, and wrapping models in Claude Code or Codex agents improved scores on a 388-task subset but barely dented 2D-occlusion-read-as-3D-alignment mistakes. The benchmark, scores and human baseline come from the releasing organizations and have not been independently reproduced.

   新智元 · 🔥 0

Experience and discussion from the community

Share my Humanity's Sixth Sense experienceAsk about Humanity's Sixth Sense