KubeRay

Headlines · 3

1. 3d
   AWS shows distributed PyTorch training with Ray Train DLCs on EKS

   AWS Machine Learning Blog provides a walkthrough for fine-tuning Qwen2.5-1.5B across eight NVIDIA L4 GPUs on two g6.12xlarge instances using Amazon EKS, KubeRay, Ray Train, PyTorch FSDP, and EFA. AWS says its Ray Train Deep Learning Container packages and patches the required CUDA, PyTorch, NCCL, EFA, and Ray components, while the sample uses one RayCluster manifest and requires no custom image build. AWS reports that enabling the EFA path doubled throughput over TCP with unchanged code, though the performance claim comes from the vendor.

   AWS Machine Learning Blog · 🔥 0
2. Oct 20
   AWS fine-tunes GPT-OSS-20B for multilingual reasoning

   AWS Machine Learning Blog reports that supervised fine-tuning raised GPT-OSS-20B’s reasoning-language accuracy from 22% to 100% on a 50-case test covering English, French, German, Spanish and Italian. Applying GRPO after SFT increased answer-language accuracy from 62% to 70%, while a Qwen2.5-7B control experiment added 6 points on multilingual reasoning but produced no gain on GSM8K.

   AWS Machine Learning Blog · 🔥 0
3. Oct 12
   AWS shows multi-node inference with Ray LLM Deep Learning Containers

   An AWS Machine Learning Blog post walks through deploying one large language model across two GPU nodes on Amazon EKS using the multi-node Ray LLM Deep Learning Container, with separate prefill and decode nodes behind a single OpenAI-compatible endpoint. The container ships Ray Serve LLM, NVIDIA's NIXL library and the Elastic Fabric Adapter stack, and the example uses Qwen3.5-9B to show prefill/decode disaggregation configured entirely through YAML with no application code. The post also warns that NIXL silently falls back to a TCP-speed transport when it cannot use libfabric, so it recommends checking worker logs to confirm the backend.

   AWS Machine Learning Blog · 🔥 0

Experience and discussion from the community

Share my KubeRay experienceAsk about KubeRay