tts-bench round three compares 91 open TTS models and 28 hosted APIs
The tts-bench author published the third round of a revamped text-to-speech benchmark covering 91 locally run open models and 28 hosted APIs on separate boards. It provides audio samples and measures time to first audio, real-time factor, UTMOS, WER, voice-cloning similarity, and API pricing, while noting that these objective metrics do not fully capture human preference or overall latency.
The broad model and API coverage gives developers a centralized way to compare locally deployed and hosted TTS options, although the results were published by the benchmark’s author and the shared Python harness was described only as “sameish.”
This is the third round of tts-bench. Thanks to everyone who gave feedback on the first two posts:
tts-bench runs every model through the sameish Python harness on the same prompts, so you can compare them fairly and actually listen to the output. This round adds a separate board for hosted APIs.
What's in it
- 91 open models run locally on an RTX 5090 32GB, an RTX 3090 24GB, and an Apple M4 16GB (sometimes)
- 28 hosted APIs on their own board, tested with the same 6 prompts (medians over several days of warm calls from the Eastern US)
- Prompts in English, Spanish and French
- Audio samples for every model, plus time to first audio, realtime factor, UTMOS, WER, voice-cloning similarity, and price for the APIs
Some results from the open board
#tts-bench#Vaniq-Edge#Maya1#Underdog Woof#Phonon 31m#Chatterbox#LongCat-AudioDiT 3.5B#FireRedTTS3#AssemblyAI#Gradium#Palabra#VUI Labs Luna