Modeling collective communication for inference
A report on inference infrastructure models collective communication latency as synchronization steps, startup latency and bytes transferred per card, then relates the results to NVLink, RDMA, NUMA and PCIe topology. It compares ReduceScatter, AllGather, AllReduce, AllToAll and other primitives, and presents model estimates plus reported H20 measurements for intra- and inter-machine communication. The H20 topology references a public SGLang machine matrix; the supplied report does not describe independent replication of its measurements.
The model gives deployment teams a structured way to compare communication algorithms, distinguish per-card traffic from system-wide egress, and identify GPU–NIC affinity issues. The report notes that small messages can favor fewer synchronization steps, while large messages are constrained mainly by effective bandwidth and cross-machine traffic.
把集合通信拆成步数、启动时延和每卡字节数，再结合 NVLink、RDMA、NUMA 与 PCIe 拓扑解释实测差异。
本文为「推理 INFRA 方法论」系列第 2/8 篇。发布日期按 2026 年 1–11 月编排；内容统一整理于 2026-11-01，包含年内后续更新的模型与实验资料。
常见通信算法
| 类别 | 原语 / 算法 | 语义或机制 | 每卡字节 | 步数 | 备注 |
|---|---|---|---|---|---|
| 多对多 | ★ ReduceScatter | 各卡拿到 1/P 的规约结果 | (P−1)/P·S ✓下界 | P−1（ring） | 每卡各发，P 张网卡并行 |
| ★ AllGather | 各卡拿到全量拼接 | (P−1)/P·S ✓下界 | P−1（ring） | = ReduceScatter 的逆向 | |
| ★ AllReduce | 各卡拿到全量规约 | 2(P−1)/P·S ✓下界 | 2(P−1)（ring） | = RS + AG，见下方变体 | |
| ★ AllToAll | 每卡给每卡发不同数据（分块转置） | (P−1)/P·S ✓下界 | 1（全对并发） | 字节由语义锁死，省不掉 | |
| 单端点 | Broadcast | root 发给所有卡，内容相同 | root 出 (P−1)·S（朴素） 每卡收 S | log₂P（树） | 朴素实现 root 出口瓶颈 |
| Scatter | root 切片发给各卡，内容不同 | root 出 (P−1)/P·S 每卡收 S/P | 1 | 字节数同 RS，但全压一张网卡 | |
| Gather | 各卡数据汇到 root 拼接 | root 入 (P−1)/P·S 每卡发 S/P | 1 | incast：P−1 卡同时打 root | |
| Reduce | 各卡数据汇到 root 规约 | 每卡发 ≈S | log₂P（树） | 规约计算集中在 root |