Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

CogGym tests 50 language models on 258 psychology experiments

AI summary

According to 掘金 · 人工智能, the MIT-led CogGym project standardized 258 cognitive-science experiments in a task-independent EML format and tested 50 language models against recorded human responses. The best model reached R² scores of 0.59 for text, 0.58 for images, and 0.43 for video, compared with human reliabilities of 0.93, 0.95, and 0.92. Performance improved with model scale, but common-sense reasoning advanced more slowly than mathematical and programming capabilities, while repeated sampling produced overly sharp response distributions.

In-depth insight

CogGym provides a quantitative way to compare model behavior with human responses, showing that strong formal-reasoning scores do not imply comparable performance on common-sense tasks.

氛围图

💡 一句话总结:数学考试 AI 已经超越人类,但「理解人类怎么想」这件事呢?MIT 团队把认知科学几十年积累的 258 个经典实验标准化成统一考卷,让 50 个大模型和人类同场竞技:模型确实越来越像人,但天花板还有很远——而且「常识」这一科的成绩增长明显掉队于数学和编程。所有「AI 达到人类水平」的说法,都该先过这张考卷。

导语:「像人」不是感觉,是可测量的科学问题

「GPT 像人一样思考吗?」——这类争论通常止于各说各话的感觉。认知科学的回答方式一直很明确:拿实验来。几十年的人类行为研究留下了成千上万个精心设计的实验范式:人怎么推断他人意图、怎么理解模糊指令、怎么看图推理因果。

#CogGym#MIT#Harvard University#Cornell University#Stanford University#EML

Original · 3 min readRead the original
Group chat

No comments yet. Start the conversation.