mini-sglang

Headlines · 2

1. Nov 3
   FreeToken targets local MoE inference on consumer hardware

   A technical report describes FreeToken as an edge-native MoE serving engine that combines GPU compute, CPU compute and host memory in a unified inference pool. The report says the project can run 290B-parameter open-weight models at interactive speeds on gaming PCs and supports features including semantic checkpoints, expert caching and dynamic VRAM allocation. It also reports support for Windows and Linux, Anthropic- and OpenAI-compatible APIs, several MoE models and NVIDIA RTX 30-, 40- and 50-series GPUs.

   掘金 · 人工智能 · 🔥 0
2. Oct 11
   SGLang trends on GitHub's Python list

   SGLang is an open-source inference framework hosted by LMSYS for large language, vision-language and diffusion models, optimized for agentic workloads, large-scale serving and RL rollouts; the repository also includes SGLang Diffusion, its built-in image and video generation engine. It supports a wide range of GPUs, TPUs, NPUs, CPUs and Apple Silicon, including NVIDIA B200/B300 and H100, AMD Instinct, Google TPU, Huawei Ascend, and ships via Docker images or a uv install. The project offers a Cookbook, docs and learning resources such as Mini-SGLang, and long-term contributors can apply for coding-agent sponsorship covering Cursor, Claude Code or OpenAI Codex.

   GitHub Trending · Python · 🔥 0

Experience and discussion from the community

Share my mini-sglang experienceAsk about mini-sglang