trn2.48xlarge

Headlines · 2

1. 6d
   RunAnywhere opens Trainium2 engine for GLM-5.3-Flash

   RunAnywhere developer said the company open-sourced an inference engine that serves the 320B-parameter, 1M-context GLM-5.3-Flash model on one AWS Trainium2 trn2.48xlarge. The project uses fused NKI layer megakernels to reduce a batch-8 decode step from 32.2 ms to 24.2 ms on the author's layout, with reproduction instructions and detailed profiles included. RunAnywhere also reported up to 28.9 agent requests per minute on the self-run SemiAnalysis AgentX benchmark, but the post provides no stock-AWS or GPU baseline.

   r/MachineLearning · 🔥 0
2. Oct 21
   AWS walkthrough: training Qwen3 on Trainium with torchtitan

   An AWS Machine Learning Blog post walks through training Qwen3 on a single trn2.48xlarge Trainium instance using torchtitan, saying TorchNeuron registers Trainium as a native PyTorch device so torchtitan can use it through standard PyTorch interfaces. The post describes three configurations that share the same torchrun launch pattern and config structure: Hugging Face Transformers v5 as a modeling backend running Hub models through AutoModelForCausalLM with published safetensors, torchtitan's native Qwen3 selected by changing config.model_spec, and an NKI mixture-of-experts path using kernels from the torchtitan-neuron recipes attached via torchtitan's override registry. AWS says the setups support both pre-training from random initialization and fine-tuning published weights, and pins versions including torch 2.14.0, torch_neuronx 2.14.3, transformers 5.16.0 and torchtitan-neuron release-2.33.

   AWS Machine Learning Blog · 🔥 0

Experience and discussion from the community

Share my trn2.48xlarge experienceAsk about trn2.48xlarge