ConvFill pairs a 0.6B talker with a large reasoner for faster voice agents
AI summary
ConvFill is a voice-agent architecture developed by students from the University of Washington UbiComp Lab and presented at EMNLP. A 0.6B talker model uses interim “cheat notes” from a large reasoner model to respond quickly while the reasoner continues processing, then reads its completed response to sustain the conversation. The r/LocalLLaMA report’s author helped build a public demo but said the talker model would need to become smarter for the approach to work as intended.
In-depth insight
ConvFill offers a concrete approach to narrowing the speed-versus-reasoning tradeoff in voice agents without requiring the small talker model to perform the full reasoning task itself.
| Voice agents are either fast or smart. ConvFill proposes a way to bridge that gap by training a smol talker model (0.6B!) to talk with cheat notes from a large reasoner model. The way it works is this: after the user's speech is transcribed, the talker replies quickly to buy itself time while the reasoner thinks. The reasoner then generates an appropriate response, which the talker reads to continue the conversation. It's a bit like if you went on a panel with rocket scientists and had a little earbud with your own expert that could tell you what to say. You just reply quickly so as not to blow your cover, and then you begin saying what the expert in your earbud is telling you. Playing with the model, I think the main idea is there and this could be a great direction to go, but the talker model should be a bit smarter for it to work like magic. Still, really cool direction! ConvFill was developed by students from the University of Washington UbiComp Lab. I found it cool, so I reached out to them so we could build a really good demo for it together. They are presenting their paper at EMNLP this week. Go try it out in our demo: https://huggingface.co/spaces/smolagents/convfill-speech-demo submitted by /u/futterneid [link] [comments] |