AWS Trainium3

Headlines · 2

1. Oct 23
   Trainium3 Advances Native PyTorch Training, but CUDA Gap Remains

   SemiAnalysis reports that Anthropic is running production inference and reinforcement-learning rollouts on AWS Trainium2 and Trainium3, while AWS Neuron is moving Trainium toward native PyTorch training with eager mode, torch.compile and standard APIs. Its engineers found the Trainium3 direction promising after testing the TorchTitan and TorchNeuron stack, but encountered unspecified bugs, long compilation times and a platform that remains far less mature than NVIDIA's CUDA ecosystem. AWS says upcoming TorchNeuron releases will use precompiled NKI kernels and a redesigned compile stack to target faster time to first step, though that target has not yet been demonstrated.

   SemiAnalysis · 🔥 0
2. Oct 23
   SGLang launches beta plugin for AWS Trainium inference and RL

   The SGLang Team, in collaboration with AWS, announced a beta plugin for running LLM inference serving and reinforcement learning rollouts on AWS Trainium2 and Trainium3. Built on TorchNeuron, the beta initially covers Qwen3, Qwen3-MoE-30B-A3B, DeepSeek-V2-Lite, GLM-4, and GLM-5.2/5.3, with parallelism, Radix Cache, BF16, FP8, and an asynchronous scheduler. In an early end-to-end test, SGLang generated Qwen3-MoE-30B-A3B rollouts on Trainium3 while Miles trained a GRPO policy with Megatron on one H200 node; performance work remains ongoing.

   LMSYS Blog · 🔥 0

Experience and discussion from the community

Share my AWS Trainium3 experienceAsk about AWS Trainium3