Inside Mina’s Real-Time Voice Meeting Assistant
The report explains how the authors built Mina, their meeting assistant for natural turn-taking, follow-up questions, interruptions, calendar queries, and business-tool execution. Mina uses server-side wake-word detection with sherpa_onnx.KeywordSpotter, binds requests to a speaker and interaction window, and keeps server VAD while leaving response and interruption decisions to the application. Tool calls are validated and executed by the application; results are returned to Qwen before an explicit continuation is requested, while long-running work can be delegated to a background MeetingAgent and tracked by task ID. The current code defaults to qwen3.8-omni-flash-realtime, whereas the cited Speech-to-Speech leaderboard evaluated Qwen Audio 3.0 Realtime Plus. The authors say the connected model still needs validation with meeting samples. The implementation also includes latency monitoring across turn waiting, tool execution, continuation, playback preparation, and interaction quality.
The report provides a concrete implementation blueprint for controlling a realtime voice agent, including wake-word handling, speaker gating, application-managed turn taking, function calling, long-running tasks, and latency measurement. Its performance claims remain unvalidated for the currently connected model.
上一篇讲了模型接入。这一篇继续处理接入之后的问题:从用户开口,到助手执行工具、返回答案,再到下一轮追问,中间需要哪些环节。
文章先讲通用机制,再对照框架与模型接口,最后回到我们自己的实现 Mina,说明每个选择的依据。
背景:先选实时语音路线,再看能否落地
我们要做一个能参与会议的助手,支持自然接话、等待补充、接受打断,以及查询日历和调用业务工具。选模型时,需要同时看两类能力:
- 语音交互:听懂输入、控制接话时机,并持续生成自然的语音。
- 业务执行:生成可靠的调用参数,让应用完成查询,再根据结果继续回答。
语音榜单为什么让我们优先看 Realtime
我们参考 Speech-to-Speech 榜单,评估模型的语音对话能力。Artificial Analysis 将语音推理、交互节奏、Agent 任务表现和响应速度分别展示。
#Mina#Qwen#Qwen Audio 3.0 Realtime Plus#qwen3.8-omni-flash-realtime#SeedRealtime#Seeduplex#Agora#Pipecat#LiveKit Agents#TEN Framework#sherpa-onnx#Artificial Analysis