Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

tts-bench round three compares 91 open TTS models and 28 hosted APIs

AI summary

The tts-bench author published the third round of a revamped text-to-speech benchmark covering 91 locally run open models and 28 hosted APIs on separate boards. It provides audio samples and measures time to first audio, real-time factor, UTMOS, WER, voice-cloning similarity, and API pricing, while noting that these objective metrics do not fully capture human preference or overall latency.

In-depth insight

The broad model and API coverage gives developers a centralized way to compare locally deployed and hosted TTS options, although the results were published by the benchmark’s author and the shared Python harness was described only as “sameish.”

This is the third round of tts-bench. Thanks to everyone who gave feedback on the first two posts:

tts-bench runs every model through the sameish Python harness on the same prompts, so you can compare them fairly and actually listen to the output. This round adds a separate board for hosted APIs.

What's in it

  • 91 open models run locally on an RTX 5090 32GB, an RTX 3090 24GB, and an Apple M4 16GB (sometimes)
  • 28 hosted APIs on their own board, tested with the same 6 prompts (medians over several days of warm calls from the Eastern US)
  • Prompts in English, Spanish and French
  • Audio samples for every model, plus time to first audio, realtime factor, UTMOS, WER, voice-cloning similarity, and price for the APIs

Some results from the open board

#tts-bench#Vaniq-Edge#Maya1#Underdog Woof#Phonon 31m#Chatterbox#LongCat-AudioDiT 3.5B#FireRedTTS3#AssemblyAI#Gradium#Palabra#VUI Labs Luna

Original · 2 min readRead the original
Group chat

No comments yet. Start the conversation.