AWS shows distributed PyTorch training with Ray Train DLCs on EKS
AWS Machine Learning Blog provides a walkthrough for fine-tuning Qwen2.5-1.5B across eight NVIDIA L4 GPUs on two g6.12xlarge instances using Amazon EKS, KubeRay, Ray Train, PyTorch FSDP, and EFA. AWS says its Ray Train Deep Learning Container packages and patches the required CUDA, PyTorch, NCCL, EFA, and Ray components, while the sample uses one RayCluster manifest and requires no custom image build. AWS reports that enabling the EFA path doubled throughput over TCP with unchanged code, though the performance claim comes from the vendor.
The walkthrough offers a concrete, reusable setup for multi-node PyTorch fine-tuning without assembling and rebuilding a GPU training image for every cluster.
What usually stops PyTorch training from scaling out to a cluster is the container, not the model code. Every container needs the same CUDA runtime, a PyTorch build compiled against it, NCCL (NVIDIA Collective Communications Library), the Elastic Fabric Adapter (EFA) userspace libraries, the AWS OFI NCCL plugin that connects NCCL to EFA, and a Ray version the cluster operator agrees with. Building that image and rebuilding it each time one of those pieces moves is a project of its own, and it has nothing to do with the model you set out to train. The Ray Train Deep Learning Container (DLC) is that image, built by AWS. AWS assembles the stack, tests it as one unit, publishes it in Amazon Elastic Container Registry (Amazon ECR) Public and keeps it patched.
Figure 1: One EKS cluster with a control plane managed by AWS, a CPU system node (m7i.xlarge) running the KubeRay operator and Ray head, and two EFA-enabled g6.12xlarge nodes in one Availability Zone. KubeRay builds it from one RayCluster manifest, Ray Train places eight workers, one per GPU, and inter-node collectives cross over EFA.
#Ray Train Deep Learning Container#AWS#Ray Train#KubeRay#PyTorch#Amazon EKS#Elastic Fabric Adapter#Qwen2.5-1.5B#NVIDIA L4