Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

HICO-Bench scores social agents on verifiable plans, not just fluent chat

AI summary

The paper introduces HICO-Bench, a benchmark of 180 everyday coordination scenarios such as resource allocation, scheduling and authorization, in which two agents holding private information converse and then independently record the plan they agreed on. A program checks Feasibility (the two records match and satisfy both participants' private conditions), while an LLM judge checks Integrity (the conversation supports the recorded plan); passing both counts as Verified Success. Across thirteen LLMs, the authors report that 36% of conversations rated highly for goal completion fail Verified Success. The paper also proposes Paired-Credit Policy Optimization (PaCPO), which asks the agent for a tentative plan after each turn and checks it against both versions of the scenario; the authors report it raises Qwen3-8B's Verified Success from 20.4% to 28.3%.

In-depth insight

The authors argue that existing social-agent evaluations mainly rely on LLM judges that rate the conversation without checking the resulting plan, which this benchmark aims to address with programmatically verifiable outcomes.

Computer Science > Computation and Language

arXiv:2610.20195 (cs)

[Submitted on 14 Oct 2026]

Title:What Makes a Social Agent's Interactions Effective? Learning from Verifiable Outcomes in Paired Worlds

Authors:Caixin Kang, Ke Wang, Yuchuan Wu, Xiaoqiang Wang, Ling Li, Bingxue Qiu, Yongbin Li, Yoichi Sato

View a PDF of the paper titled What Makes a Social Agent's Interactions Effective? Learning from Verifiable Outcomes in Paired Worlds, by Caixin Kang and 7 other authors

#HICO-Bench#PaCPO#Qwen3-8B

Original · 3 min readRead the original
Group chat

No comments yet. Start the conversation.