TraceYourThinking

Headlines · 1

1. Oct 11
   MIT's HugAgent tests whether AI 'digital twins' can predict people in new scenarios

   Researchers at MIT and other institutions propose HugAgent, a benchmark that uses interviews with 54 real participants to test whether large models can portray a specific person—not just restate current views but predict how that person would answer previously unseen hypothetical scenarios on universal healthcare, public surveillance and housing densification, across 1,742 evaluation items; the paper was accepted as an EMNLP 2026 Main Oral. Models did best at inferring a person's current beliefs (top accuracy 77.56%; GPT-4o 74.66%, against an 84.84% two-week self-consistency ceiling), while on the belief-dynamics update task the best result was 68.61% (Claude Sonnet 4.5), then LLaMA 3.3 70B at 67.57% and GPT-4o at 63.11%, against an 85.66% human ceiling. Personal interviews clearly helped prediction (GPT-4o rose from 39.83% to 63.11%), but adding more interview rounds did not consistently improve forecasts for new situations.

   虎嗅 AI · 🔥 0

Experience and discussion from the community

Share my TraceYourThinking experienceAsk about TraceYourThinking