PyTorch 2.15发布，强化分布式与低精度推理
PyTorch Blog宣布发布PyTorch 2.15，重写的NCCL后端现为所有NCCL进程组的默认选项，同时保留旧版回退路径。新版本加入单边torch.distributed传输、FSDP2与流水线控制、NVIDIA Blackwell上的NVFP4解码优化，以及Apple Silicon的FP8和解码改进，并扩展Python 3.11至3.15、CUDA 13.2与13.4、ROCm 10.0、AMD及Intel GPU支持。
PyTorch Blog称PyTorch 2.15对分布式训练、低精度推理、硬件适配和Python兼容性进行了重要更新。
We are excited to announce the release of PyTorch® 2.15 (release notes)!
The PyTorch 2.15 release features the following changes:
- Rewritten NCCL backend becomes the default, with a flight recorder that dumps on hangs and
TORCH_DIST_USE_NCCL2=0ornccl-legacyas rollback paths - One-sided transports for torch.distributed, with a NIXL backend and CUDA-graph-capturable transfers
- FSDP2 and pipelining gain finer memory and communication control, with fp32 for precision-sensitive parameters, CUDA-graph-captured gradient accumulation, and gradient reduction overlapped with compute
- Faster 4-bit (NVFP4) LLM decoding on NVIDIA Blackwell GPUs through fewer, better-overlapped kernel launches in NVGEMM
- Apple Silicon gains FP8 and faster decode, with
float8_e4m3fnand_scaled_mm, a new grouped_mm with a decode fast path (1.32–1.34x tokens/s on MoE decode), and publictorch.mps.StreamAPIs - Broader platform support with FlyDSL kernels on AMD gfx950, Tensor Data Mover support for gfx1250 (MI450), ROCm 10.0 wheels, simpler SYCL kernel authoring and multi-GPU profiling on Intel GPUs, CUDA 13.2 as the default wheel with CUDA 13.4 alongside it, and torch.compile on Python 3.15
This release is composed of 3,117 commits from 583 contributors since PyTorch 2.14. We want to sincerely thank our dedicated community for your contributions. As always, we encourage you to try these out and report any issues as we improve 2.15. More information about how to get started with the PyTorch 2-series can be found at our Getting Started page.
#PyTorch 2.15#PyTorch#NCCL#NIXL#FSDP2#NVGEMM#MPS#CUDA 13.2#CUDA 13.4#ROCm 10.0#Python 3.15#NVFP4