sglang-omni: A multi-stage serving runtime for omni, speech, and TTS models.
What problem it solves Interactive multimodal and speech systems often require generation to span heterogeneous stages with different compute patterns, dependency structures, and resource needs. Serving these pipelines requires coordinated stage lifecycles, inter-stage transport, model integrations, and a unified API surface. Key features Coordinated multi-stage runtime — Models generation as coor…
github.comhttps://github.com/sgl-project/sglang-omni