Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

SGLang adds native support for Reflection's Beam, a 501B-parameter MoE

AI summary

SGLang's team writes that Reflection's Beam is a 501B-parameter mixture-of-experts model with 23B active parameters per token, aimed at software engineering and long-horizon agentic tasks and slated for an Apache 2.0 release. SGLang now supports Beam natively and powers Reflection's production API, with BF16, NVFP4 and dynamic FP8 checkpoints plus DFlash2 speculative decoding, prefix caching/HiCache and chunked prefill. The reported figures: replaying an OpenHands multi-turn agentic coding workload with the NVFP4 checkpoint on four NVIDIA GB300 GPUs, SGLang reached 127.6K total tokens/sec per GPU at 185.87 tokens/sec per user (p50) with concurrency 32.

In-depth insight

If Beam ships under Apache 2.0 as planned, it would add a very large MoE option for coding agents, and serving-stack maturity will shape real deployment cost.

ReflectionBeamSGLang

Original · 14 min readRead the original
Group chat

No comments yet. Start the conversation.