SGLang-Neuron v0.1.0登陆AWS Trainium
AI 总结
RadixArk与Amazon Annapurna Labs发布SGLang-Neuron v0.1.0,这是一款原生基于PyTorch的开源后端,可在AWS Trainium上服务语言模型和扩散模型;LMSYS Blog称其处于公开预览,AWS Machine Learning Blog则称已公开可用。该插件沿用SGLang服务器接口,并加入异步运行时、megakernels、预填充与解码分离以及基于主机内存的HiCache。厂商披露的配置包括跨三个Trainium3节点运行Kimi-K3、支持100万token上下文的GLM-5.2,以及在Trainium2上运行的SGLang Diffusion。
深度见解
此次发布让开发者可以通过熟悉的SGLang API和容器工作流,在AWS Trainium上运行模型服务及强化学习rollout。
On this page
- TL;DR
- Introduction
- How it works
- One asynchronous runtime
- Megakernels: one kernel per step
- Prefill-decode disaggregation over EFA
- Kimi-K3 on three Trainium3 nodes
- GLM-5.2 at 1M context
- HiCache: a DRAM tier under the KV cache
- SGLang Diffusion on Trainium
- Feature and model coverage
- RL rollouts with Miles
- Get started
- What's next
- Community
- Acknowledgments
#SGLang-Neuron#SGLang#AWS Trainium#RadixArk#Amazon Annapurna Labs#TorchNeuron#GPT-OSS-120B#Kimi-K3#GLM-5.2#SGLang Diffusion#AWS Deep Learning Container