SteerTTS: a 23M-parameter English TTS steered by numeric controls
Developer WhissleAI posted SteerTTS on r/MachineLearning, a 23M-parameter English TTS system combining a 9.6M-parameter acoustic model with a Vocos vocoder and 12 numeric z-score controls for rate, loudness, pitch and related attributes. The post reports RTFx of about 60–80 on an RTX 5070 and 4–8 on CPU, plus 2,825 built-in voices, 51 style tags and sentence-level streaming. The developer also notes weak pause and voicing controls, limited emotion-tag separation, low cloning similarity and no human evaluations yet; weights are CC BY-NC 4.0 and code is MIT.
The developer says it was built to fill a gap in small, steerable TTS models that can be adjusted contextually with controls supplied by an LLM during a conversation.
I was looking for small TTS model which is steerable but couldn't find any, specially something I can adjust contextually by providing controls from LLM during a conversation. So I built one.
What it is:
- 9.6M-param acoustic model (attention + bidirectional Mamba-3, FastPitch-style) + Vocos vocoder = 23M total
- 12 controls, each a z-score relative to the chosen voice’s own neutral speech: rate, loudness, median pitch, pitch range, final rise/fall, pauses, spectral tilt, breathiness-related measures.
- Also: 51 style tags, free-text prompts (resolved into an inspectable JSON request), inline markup (
*emphasis*,[pause 400ms],<slow>…</slow>), sentence-level streaming - 2,825 built-in (anonymized) voices, plus experimental zero-shot cloning
Speed:
- RTFx ~60–80 on an RTX 5070 - ~4–8 on CPU without the optional flow refiner. First streamed audio in ~0.5 s on CPU.
Quality: