Miles

Headlines · 3

1. 1d
   SGLang and Miles add day-0 support for Muse Spark Open

   An LMSYS blog post says SGLang and Miles have added day-0 support for Muse Spark Open, an open-weight multimodal mixture-of-experts model. Its text backbone has 816B parameters with roughly 42B active parameters per token, accepts text, image, video and audio inputs, supports a 1M-token context window, and ships NVFP4 and BF16 checkpoints. The SGLang integration adds sequence parallelism with NVFP4 MegaMoE for prefill, multimodal preprocessing optimizations and DFlash V2 speculative decoding; the team reports that on 8 NVIDIA B300 GPUs with the NVFP4 checkpoint, an OpenHands multi-turn agentic coding replay reaches 14.8K total tokens/sec per GPU at concurrency 1 and 80.3K at concurrency 128.

   LMSYS Blog · 🔥 5
2. 3d
   Modal: mixed-precision, mixed-accelerator samplers enable stable async RL

   In a technical blog post, Modal reports that running reinforcement-learning samplers on mixed precisions and mixed accelerators can keep large-model RL stable while cutting cost. Six kinds of accelerators feed a single BF16 trainer on B200: B200 and B300 serve NVFP4 weights, H100 and H200 serve FP8 and BF16 weights, and A100 and RTX PRO 6000 serve BF16 weights, each running its own kernels. The run trains Qwen3.6-35B-A3B on agentic coding with up to 256K tokens of context, using the 2.7K repository tasks Xiaomi released with MiMo-V2.6, and evaluates on SWE-bench Pro V2 and its HARD-51 subset. The authors add masked importance sampling, score centering and top-p sampling with mask replay as off-policy corrections; uncorrected GRPO collapses soon after training starts on the mixed samplers, while SC+MIS with or without top-p trains stably, with pass@4 rising over roughly 100–140 steps and levelling around 65–75% on HARD-51 and rising from 89% to about 94% on the full set, at about 15% lower sampler cost per update for most of training.

   Modal Blog · 🔥 0
3. Oct 9
   SGLang and Miles tuned for NVIDIA Vera Rubin, serving Kimi K3 NVFP4

   The SGLang & Miles team says it received early access to two NVIDIA Vera Rubin nodes (8 GPUs) and brought both SGLang inference and Miles RL training up on the platform. For serving the Kimi K3 NVFP4 checkpoint, it retuned attention, MoE/communication and KDA kernels, reporting a 16% faster deeper MLA pipeline at batch 16 and 128k context, a 20% faster split-KV reduction, a 5.9% end-to-end gain from fusing the MoE tail, and 276 fewer kernel launches per decode step. On the training side, Miles uses SGLang for rollout and Megatron for training, and ran end-to-end RL out of the box on a single 4-GPU tray with Qwen3-30B-A3B and DeepSeek-V4-Flash, plus agentic RL with 64 sandboxes on the Arm-based Vera CPU on SWE-bench Verified. The team also cites Cognition as reporting up to a 4.8x increase in total token throughput on Vera Rubin NVL72 versus GB200 NVL72 using its own private SGLang fork.

   LMSYS Blog · 🔥 0

Experience and discussion from the community

Share my Miles experienceAsk about Miles