SGLang launches open-source PyTorch-native TPU serving backend
SGLang launched SGLang-TorchTPU, an open-source native PyTorch backend that brings its serving stack to Google Cloud TPU while retaining the same core codebase used for GPUs. The project, developed by RadixArk’s SGLang team with Google and Meta, includes real-TPU CI tests and support for multiple language and diffusion models. SGLang and Meta personnel said the backend is running production inference at Meta, while Tom’s Hardware also reported the open-source release of a matching PyTorch-native vLLM version.
SGLang-TorchTPU gives PyTorch-based serving deployments a native path onto Google TPUs, with the project and Meta representatives confirming production use.
On this page
- TL;DR
- Introduction
- How it works
- Feature and model coverage
- Reliability
- Training with TPU on Miles
- Built with Google and Meta
- Get started
- Community
- Acknowledgments
#SGLang#SGLang-TorchTPU#RadixArk#Google#Meta#PyTorch#TorchTPU#Google Cloud TPU#vLLM#vLLM-TorchTPU