audio.cpp新增Kokoro与OpenVoice语音克隆配方
AI 总结
一名开发者在 audio.cpp 中加入了 any-tts 配方，将 Kokoro 与 OpenVoice ToneColorConverter 配对，在无需重新训练 TTS 模型的情况下组合语音合成与声音转换。开发者称其峰值显存额外增加约 600 MB，CUDA 速度下降约 10%，CPU 速度约为原来的一半，但仍可达到两倍实时。开发者称 Kokoro 的结果“相当不错”，但这一质量评价未经过独立验证。
深度见解
该配方为现有 TTS 模型加入目标说话人声音转换及语音克隆能力提供了一种无需重新训练的实现方式，并记录了具体的显存和性能开销。
While looking for audio models, I came across OpenVoice. It's getting pretty old, but surprisingly still ranks high on TTS leaderboards.
What caught my attention was its design: essentially TTS + voice conversion. The ToneColorConverter can run independently of the TTS model.
That got me thinking: Can we pair it with other TTS models?
I tried it with Kokoro, and the results were pretty good!
So I added a small recipe any-tts to audio.cpp for Kokoro + OpenVoice ToneColorConverter. Recipes were introduced in v0.9.2 as predefined pipelines that combine multiple audio models.
The overheads:
You can also try pairing any tiny TTS model with ToneColorConverter. The TTS model determines the speech content, pronunciation, and prosody, while the VC model transfers the target speaker's voice characteristics. The built-in voice you choose can also affect the final output quality.
A simple way to give existing TTS models voice cloning capabilities without retraining them! submitted by /u/Acceptable-Cycle4645
[link] [comments] |