IIT Madras

Headlines · 1

1. 18h
   NVIDIA and AI4Bharat to release 2.8T-token Indic pretraining corpus and open pipeline

   NVIDIA and AI4Bharat, IIT Madras' Indian-language research lab, are releasing Nemotron-pretraining-Indic-v1, a corpus of roughly 2.8 trillion tokens across the 22 languages listed in India's Constitution — 2,601.0 billion native-script tokens and 224.6 billion romanized tokens under the Gemma-4 tokenizer, spanning 34 language-script configurations. It was built from 15 Nemotron-CC-v2 subsets with NeMo Data Designer running document-level translation, MQM-style quality scoring with a 75/100 threshold and targeted repair, while retaining source identifiers so each output traces back to a real document; NVIDIA is also open-sourcing the GPU-accelerated generation pipeline. Continued pretraining for Nemotron 3.5 Super VL added 10 billion Indic tokens, 10% of a 100-billion-token budget, and the technical report reports gains of 4.98 to 9.82 points on five Indic knowledge and reasoning benchmarks, though that comparison mixes synthetic and natural Indic data.

   Hugging Face · 🔥 7

Experience and discussion from the community

Share my IIT Madras experienceAsk about IIT Madras