NVIDIA and AI4Bharat to release 2.8T-token Indic pretraining corpus and open pipeline
NVIDIA and AI4Bharat, IIT Madras' Indian-language research lab, are releasing Nemotron-pretraining-Indic-v1, a corpus of roughly 2.8 trillion tokens across the 22 languages listed in India's Constitution — 2,601.0 billion native-script tokens and 224.6 billion romanized tokens under the Gemma-4 tokenizer, spanning 34 language-script configurations. It was built from 15 Nemotron-CC-v2 subsets with NeMo Data Designer running document-level translation, MQM-style quality scoring with a 75/100 threshold and targeted repair, while retaining source identifiers so each output traces back to a real document; NVIDIA is also open-sourcing the GPU-accelerated generation pipeline. Continued pretraining for Nemotron 3.5 Super VL added 10 billion Indic tokens, 10% of a 100-billion-token budget, and the technical report reports gains of 4.98 to 9.82 points on five Indic knowledge and reasoning benchmarks, though that comparison mixes synthetic and natural Indic data.
Indic language technology has long been constrained by scarce training data, so a large open corpus plus a reusable generation recipe lowers the barrier to building models for these languages.