Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

VoxMem benchmark: audio LLMs score under 40% on multi-session speech memory

AI summary

A team from the University of Melbourne and UNSW proposes VoxMem, a multi-session speech memory benchmark built on an "acoustic evidence × memory operation" framework covering 15 combinations, with 799 questions, 3,196 evaluation instances and 34,743 spoken sessions (about 177 hours). Each question is held constant across 8K–64K token history lengths. Evaluating 15 audio LLMs, the study reports no model topping 40% overall accuracy at 32K (best: Qwen3.8-Omni-Flash at 38.5%), while average accuracy for tracking tone changes and background-sound changes was just 3.4% and 1.2%. Error analysis attributes most failures to locating and retaining acoustic evidence rather than to reasoning.

In-depth insight

By separating "what was said" from "who said it, how, and what was audible," it suggests longer context windows alone do not deliver real long-term speech memory.

新智元 2026-10-09 11:55 北京

新智元报道  

在语音助手、多用户共享的家庭助理等长期对话场景中,模型需要跨越多次会话记住用户的信息,但现有评测常遭遇两大难题:

一是语音中的关键信息不止于文字,说话人身份、语气和背景声只存在于音频信号里,转写后即丢失,而现有基准大多只考词汇内容;

二是真实交互分布在相隔一段时间的多个会话中,现有基准却局限于单段录音或单段连续对话,且在不同长度下使用不同题目,无法分离历史长度本身的影响。

#VoxMem#University of Melbourne#UNSW#Qwen3.8-Omni-Flash#Gemini#Qwen

Original · 12 min readRead the original
Group chat

No comments yet. Start the conversation.