SGLang-Neuron v0.1.0 brings SGLang serving to AWS Trainium
RadixArk and Amazon Annapurna Labs released SGLang-Neuron v0.1.0 as an open-source, PyTorch-native backend for serving language and diffusion models on AWS Trainium, with the LMSYS Blog describing it as being in public preview and AWS Machine Learning Blog calling it publicly available. The plugin retains SGLang’s server interface and adds an asynchronous runtime, megakernels, prefill-decode disaggregation, and a host-memory HiCache tier. Vendor-reported configurations include Kimi-K3 across three Trainium3 nodes, GLM-5.2 with a 1M-token context, and SGLang Diffusion on Trainium2.
The release gives practitioners an open serving stack for running SGLang models and reinforcement-learning rollouts on AWS Trainium through the familiar SGLang API and container workflow.
On this page
- TL;DR
- Introduction
- How it works
- One asynchronous runtime
- Megakernels: one kernel per step
- Prefill-decode disaggregation over EFA
- Kimi-K3 on three Trainium3 nodes
- GLM-5.2 at 1M context
- HiCache: a DRAM tier under the KV cache
- SGLang Diffusion on Trainium
- Feature and model coverage
- RL rollouts with Miles
- Get started
- What's next
- Community
- Acknowledgments
#SGLang-Neuron#SGLang#AWS Trainium#RadixArk#Amazon Annapurna Labs#TorchNeuron#GPT-OSS-120B#Kimi-K3#GLM-5.2#SGLang Diffusion#AWS Deep Learning Container