GB200 NVL72

Headlines · 2

1. 2d
   SkyPilot: topology-aware scheduling for Grace Blackwell NVL72 racks

   SkyPilot says running the same distributed training job over InfiniBand instead of NVLink can degrade performance by 43%, and that topology-aware scheduling can yield 27% more usable GPUs from the same Grace Blackwell racks. The post describes NVIDIA's NVL72 systems, which connect all 72 GPUs in a rack over a unified NVLink fabric (18 NVLink ports per GPU at 100 GB/s each, 130 TB/s aggregate internally), and how the SkyPilot Platform keeps multi-node jobs rack-local through Topology-Aware Scheduling, in-place recovery from hardware failures in Frontier Trainer, and Automatic Defragmentation of shared clusters. It also cites Meta's measured mean time between interruptions of roughly 490 days per 4-GPU node, implying about 27 days for an 18-node rack, and recommends leaving spare nodes in each rack.

   Hacker News 首页（100+ 分） · 🔥 0
2. Oct 20
   Meta details Prometheus, an Ohio supercluster heading toward 1 million GPUs

   Meta published engineering details of its Prometheus supercluster in New Albany, Ohio, saying it already runs hundreds of thousands of Nvidia GPUs across several buildings and is planned to grow to roughly 1 million operating in lockstep. The first gigawatt-scale phase is finished and was used to train Meta's Muse family of models, including Muse Wonder, on Nvidia GB200 NVL72 and GB300 NVL72 racks, with Vera Rubin NVL72 planned for a third platform. Meta says a year of engineering cut training interruptions by 99%, from nearly 1,000 per 8,000 GPUs per day in mid-2025 to about 11, and that each 72-GPU rack trains on a 64-GPU slice with eight GPUs kept as spares.

   The Next Web · 🔥 0

Experience and discussion from the community

Share my GB200 NVL72 experienceAsk about GB200 NVL72