Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

MIT's HugAgent tests whether AI 'digital twins' can predict people in new scenarios

AI summary

Researchers at MIT and other institutions propose HugAgent, a benchmark that uses interviews with 54 real participants to test whether large models can portray a specific person—not just restate current views but predict how that person would answer previously unseen hypothetical scenarios on universal healthcare, public surveillance and housing densification, across 1,742 evaluation items; the paper was accepted as an EMNLP 2026 Main Oral. Models did best at inferring a person's current beliefs (top accuracy 77.56%; GPT-4o 74.66%, against an 84.84% two-week self-consistency ceiling), while on the belief-dynamics update task the best result was 68.61% (Claude Sonnet 4.5), then LLaMA 3.3 70B at 67.57% and GPT-4o at 63.11%, against an 85.66% human ceiling. Personal interviews clearly helped prediction (GPT-4o rose from 39.83% to 63.11%), but adding more interview rounds did not consistently improve forecasts for new situations.

In-depth insight

It shows that piling up interview data about a person does not by itself make a model a reliable stand-in for them in unfamiliar situations—a key limitation for synthetic respondents and personalized agents.

只要和一个人聊得足够久,大模型就能成为他的「数字分身」吗?从虚拟用户、合成受访者,到个性化AI Agent,一条越来越流行的技术路线是:先通过访谈、问卷和个人资料了解一个真实的人,再让大模型扮演这个人,预测他在没见过的问题上会如何回答。但这里有一个更难的问题:AI即使已经学会了你现在怎么想,换一个此前没问过的......

本文来自微信公众号: 新智元 ,编辑:LRST,作者:新智元,原文标题:《EMNLP'26 Oral | 麻省理工:靠访谈让AI模拟真人,这条路能走多远?》

只要和一个人聊得足够久,大模型就能成为他的「数字分身」吗?

从虚拟用户、合成受访者,到个性化AI Agent,一条越来越流行的技术路线是:先通过访谈、问卷和个人资料了解一个真实的人,再让大模型扮演这个人,预测他在没见过的问题上会如何回答。

#MIT#HugAgent#TraceYourThinking#GPT-4o#Claude Sonnet 4.5#LLaMA 3.3 70B#Qwen2.5-32B#EMNLP 2026#Gemini#DeepSeek

Original · 13 min readRead the original
Group chat

No comments yet. Start the conversation.