TL;DR: When Coqui AI shut down, they left XTTS v2 without official training code for the DVAE audio tokenizer and Latent HiFi-GAN vocoder, and fine-tuning new scripts (Arabic, Indic, etc.) broke on fixed vocabularies. I built Open-XTTS-Trainer — an open-source, modular 3-component training suite under MPL-2.0 with verified GPU loss convergence on real human speech and a 1-click Colab demo.

Why I Built This

XTTS v2 is one of the best local zero-shot TTS models we have, but training/adapting it has been painful since Coqui closed:

• 1.Missing Component Recipes: Only GPT fine-tuning was commonly available. Stage 1 (DVAE) and Stage 3 (HiFi-GAN vocoder) training pipelines were either unreleased or fragmented.
• 2.The Tokenizer Wall: XTTS was hardcoded for 17 languages. Feeding text with new Unicode scripts (Hindi, Arabic, Thai, Dhivehi, etc.) caused token out-of-vocab or matrix dimension crashes.
• 3.Dependency Rot: Upgrading to modern PyTorch (2.4+) broke on missing torchcodec and licensing prompts.

What’s in the Suite

• •Stage 1 (Audio Tokenizer — DVAE): Full training & fine-tuning loop with Gumbel-Softmax codebook annealing and mel-spectrogram reconstruction loss.
• •Stage 2 (Text & Vocab Extender — GPT): Scans a new language corpus, discovers unique Unicode glyphs, appends them to the tokenizer, and dynamically resizes the GPT embedding matrix without corrupting existing weights. Includes the autoregressive GPT training loop.
• •Stage 3 (Neural Vocoder — Latent HiFi-GAN): Latent-conditioned vocoder training with Multi-Period (MPD) and Multi-Scale (MSD) discriminators and feature-matching loss.
• •Dependency-Rot Free: Runs out-of-the-box on modern PyTorch (Python 3.10–3.13) without torchcodec blocking.