SGLang推出AWS Trainium插件测试版
SGLang团队与AWS宣布推出测试版插件，可在AWS Trainium2和Trainium3上运行大语言模型推理服务及强化学习rollout。该版本基于TorchNeuron，初期支持Qwen3、Qwen3-MoE-30B-A3B、DeepSeek-V2-Lite、GLM-4和GLM-5.2/5.3，并提供多种并行方式、Radix Cache、BF16、FP8及异步调度器。团队还展示了Qwen3-MoE-30B-A3B在Trainium3生成rollout、Miles在单个H200节点使用Megatron训练GRPO策略的早期端到端运行，目前性能优化仍在进行。
该插件为开放模型服务和异构强化学习增加了AWS Trainium路径，使rollout生成与策略训练可以使用不同硬件。
On this page
Over the past year, we have continued to expand the hardware SGLang supports, alongside launch-day support for new open models and RL rollouts in Miles. Now, we are collaborating with AWS to bring SGLang to AWS Trainium.
As frontier AI moves toward heterogeneous hardware, AWS Trainium represents a unique opportunity. Purpose-built by AWS for AI and available at scale in the cloud, Trainium is designed to deliver strong price-performance for high-volume workloads. Open-model serving and RL rollout generation are prime examples: both run continuously, and cost per token directly determines how much capacity teams can deploy. Our goal is to make SGLang a first-class option for running these workloads on Trainium.
In close collaboration with AWS, and in conjunction with the TorchNeuron launch, we are announcing the beta of the SGLang Trainium plugin, built on the newly released TorchNeuron. The beta enables both inference serving and RL rollouts on Trainium.
#SGLang#AWS#SGLang Trainium Plugin#TorchNeuron#AWS Trainium2#AWS Trainium3#Qwen3#Qwen3-MoE-30B-A3B#DeepSeek-V2-Lite#GLM-4#GLM-5.2/5.3#Miles