InfiniBand

Headlines · 2

1. 2d
   SkyPilot: topology-aware scheduling for Grace Blackwell NVL72 racks

   SkyPilot says running the same distributed training job over InfiniBand instead of NVLink can degrade performance by 43%, and that topology-aware scheduling can yield 27% more usable GPUs from the same Grace Blackwell racks. The post describes NVIDIA's NVL72 systems, which connect all 72 GPUs in a rack over a unified NVLink fabric (18 NVLink ports per GPU at 100 GB/s each, 130 TB/s aggregate internally), and how the SkyPilot Platform keeps multi-node jobs rack-local through Topology-Aware Scheduling, in-place recovery from hardware failures in Frontier Trainer, and Automatic Defragmentation of shared clusters. It also cites Meta's measured mean time between interruptions of roughly 490 days per 4-GPU node, implying about 27 days for an 18-node rack, and recommends leaving spare nodes in each rack.

   Hacker News 首页（100+ 分） · 🔥 0
2. Oct 20
   SF Compute outlines a programmable GPU compute market

   In a Latent Space interview recorded at AI Engineer New York, SF Compute CEO Evan Conrad described a market that settles GPU-hour contracts with usable compute rather than cash against an index. He said the company now supports programmatic control of areas including networking and memory, dynamic subdivision of InfiniBand fabrics, and different pricing for interruptible batch workloads versus live inference; SF Compute also reported several hundred million dollars in volume and new clusters in development.

   Latent Space · 🔥 0

Experience and discussion from the community

Share my InfiniBand experienceAsk about InfiniBand