LFM3-Omni-2B runs locally in a browser via WebGPU
AI summary
A Reddit post from /u/xenovatech presented LFM3-Omni-2B running locally in a browser through WebGPU, with a linked Hugging Face demo available for inspection. The post claimed that custom WebGPU kernels achieved about 280 tokens per second on an Apple M4 Max, with 90 ms to the first token and 140 ms to the first audio, but these performance figures remain unverified.
In-depth insight
The reported demo offers a browser-local deployment of a 2B model that handles visual, audio and text inputs without separate ASR or TTS models, according to the post.
| The first true omni model that can run 100% locally in a browser... fully end-to-end, with no separate ASR or TTS models needed. With custom WebGPU kernels, we're able to run LFM3-Omni-2B at ~280 tok/s on an M4 Max, accepting visual, audio, or text and producing interleaved text and audio tokens. Just 90 ms to the first token and 140 ms to the first audio. Link to the demo: https://huggingface.co/spaces/LiquidAI/LFM3-Omni-2B-WebGPU submitted by /u/xenovatech [link] [comments] |