CrossProfile Exposes Memory Failures in LLM Group Decisions
A newly submitted paper introduces CrossProfile, a diagnostic benchmark with 60 templates, automatically verified answer keys, and 480 hidden-profile tasks. Testing Qwen2.5-7B, Llama-3.1-8B, and two DeepSeek models, the study finds that rarity- and relevance-based memory selection can fail systematically, while the final decision model may overvalue facts attributed to many agents. The authors also report that a complete natural discussion transcript can yield lower accuracy than using no notes.
CrossProfile provides a controlled way to test whether compressed group memories preserve useful evidence, highlighting failures caused by decoys, evidence balance, and attribution-driven social proof.
Computer Science > Artificial Intelligence
arXiv:2610.37624 (cs)
[Submitted on 27 Oct 2026]
Title:Rare, Relevant, and Still Wrong: How Memory Fails LLM Group Decisions
Authors:Anouar Imel, Imad Charradi, Abderrahman Sghir, Khalil El Khammar
View a PDF of the paper titled Rare, Relevant, and Still Wrong: How Memory Fails LLM Group Decisions, by Anouar Imel and 3 other authors