vLLM-TorchTPU

Headlines · 1

1. Oct 20
   SGLang launches open-source PyTorch-native TPU serving backend

   SGLang launched SGLang-TorchTPU, an open-source native PyTorch backend that brings its serving stack to Google Cloud TPU while retaining the same core codebase used for GPUs. The project, developed by RadixArk’s SGLang team with Google and Meta, includes real-TPU CI tests and support for multiple language and diffusion models. SGLang and Meta personnel said the backend is running production inference at Meta, while Tom’s Hardware also reported the open-source release of a matching PyTorch-native vLLM version.

   LMSYS Blog · 🔥 0

Experience and discussion from the community

Share my vLLM-TorchTPU experienceAsk about vLLM-TorchTPU