vLLM-Omni

Headlines · 1

1. 3h
   inclusionAI releases Ming-3.0-audio-flash, a 126B MoE audio-language model under MIT

   inclusionAI has published Ming-3.0-audio-flash on Hugging Face, a native audio-language model that takes audio together with text instructions and supports speech recognition, spoken question answering, audio understanding and audio reasoning, with optional synthesized-speech output. According to the model card, it is a Mixture-of-Experts model with 126B total parameters and 6.5B activated per inference step, with Chinese and English as primary languages and claimed support for accents, dialects and noisy multi-speaker recordings, released under the MIT license. The card documents Transformers, SGLang and vLLM-Omni deployment paths; the poster notes the four-GPU SGLang and eight 80GB H800 vLLM-Omni setups are reference configurations, not minimum requirements.

   r/LocalLLaMA · 🔥 7

Experience and discussion from the community

Share my vLLM-Omni experienceAsk about vLLM-Omni