PyTorch 2.15 ships new distributed backend and low-precision inference updates
PyTorch Blog announced PyTorch 2.15, making the rewritten NCCL backend the default for every NCCL process group while retaining legacy rollback options. The release also adds one-sided torch.distributed transports, FSDP2 and pipelining controls, faster NVFP4 decoding on NVIDIA Blackwell, and FP8 and decode optimizations for Apple Silicon. Platform support expands to Python 3.11–3.15, CUDA 13.2 and 13.4, ROCm 10.0, AMD, and Intel GPUs.
PyTorch Blog says PyTorch 2.15 materially updates distributed training, low-precision inference, hardware enablement, and Python compatibility across major accelerator platforms.
We are excited to announce the release of PyTorch® 2.15 (release notes)!
The PyTorch 2.15 release features the following changes:
- Rewritten NCCL backend becomes the default, with a flight recorder that dumps on hangs and
TORCH_DIST_USE_NCCL2=0ornccl-legacyas rollback paths - One-sided transports for torch.distributed, with a NIXL backend and CUDA-graph-capturable transfers
- FSDP2 and pipelining gain finer memory and communication control, with fp32 for precision-sensitive parameters, CUDA-graph-captured gradient accumulation, and gradient reduction overlapped with compute
- Faster 4-bit (NVFP4) LLM decoding on NVIDIA Blackwell GPUs through fewer, better-overlapped kernel launches in NVGEMM
- Apple Silicon gains FP8 and faster decode, with
float8_e4m3fnand_scaled_mm, a new grouped_mm with a decode fast path (1.32–1.34x tokens/s on MoE decode), and publictorch.mps.StreamAPIs - Broader platform support with FlyDSL kernels on AMD gfx950, Tensor Data Mover support for gfx1250 (MI450), ROCm 10.0 wheels, simpler SYCL kernel authoring and multi-GPU profiling on Intel GPUs, CUDA 13.2 as the default wheel with CUDA 13.4 alongside it, and torch.compile on Python 3.15
This release is composed of 3,117 commits from 583 contributors since PyTorch 2.14. We want to sincerely thank our dedicated community for your contributions. As always, we encourage you to try these out and report any issues as we improve 2.15. More information about how to get started with the PyTorch 2-series can be found at our Getting Started page.
#PyTorch 2.15#PyTorch#NCCL#NIXL#FSDP2#NVGEMM#MPS#CUDA 13.2#CUDA 13.4#ROCm 10.0#Python 3.15#NVFP4