NeMo-DCR
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- NVIDIA's NeMo-DCR: bit-exact delta refit for trillion-parameter agentic RL
NVIDIA researchers posted NeMo-DCR (Delta-Compressed Refit), a method that synchronizes policy updates between training and rollout clusters by sending only weight changes while remaining bit-exact against a dense refit. The paper reports that about 1% of BF16 training weights change stored values per step, and that at 3% and 5% change rates refits of 30B–1T models run 12–40x faster than a transport-only full-checkpoint reference; a 1T relay-tree refit at 3% takes 150 seconds versus 87.5 minutes to move a full checkpoint between two AWS regions. The code is open-sourced in NVIDIA NeMo RL PR #2444.
Hugging Face · Papers · 🔥 0
Experience and discussion from the community
Share my NeMo-DCR experienceAsk about NeMo-DCR
Nobody has shared their experience with NeMo-DCR yet.