Megatron

Headlines · 2

1. 4d
   Moore Threads Open-Sources Megatron-MUSA-Titan for Large-Scale MoE Training

   Moore Threads announced the open-sourcing of Megatron-MUSA-Titan, a large-model training infrastructure project adapted for the company’s MUSA platform and full-function GPUs. IT之家 reported that it targets trillion-parameter MoE pretraining and reinforcement learning on clusters with 10,000 GPUs. The project supports blockwise FP8 training and can work with the DeepEP and ACE asynchronous communication engines to overlap expert parallelism and computation. Moore Threads said the project had been used in continued trillion-parameter MoE pretraining on its 10,000-GPU-scale cluster and supports coordination with slime and miles, Megatron training, and SGLang sample generation. Its reported model FLOPS utilization and long-running stability were not independently verified in the supplied reports.

   IT之家 AI · 🔥 0
2. Oct 9
   Blog SGLang and Miles on NVIDIA Vera Rubin We had early access to two NVIDIA Vera Rubin nodes (8 GPUs) and used them to bring SGLang and Miles up on the platform: SGLang for inference and Miles for RL training. SGLang & Miles Team October 9, 2026

   On this page Summary Inference: Serving Kimi K3 NVFP4 Attention MoE and communication kernels KDA RL training: Miles on NVIDIA Vera Rubin Agentic RL on the Vera CPU What's Next Acknowledgements Summary We had early access to two NVIDIA Vera Rubin nodes (8 GPUs) and used them to bring SGLang and Miles up on the platform: SGLang for inference and Miles for RL training. On the inference side, we tuned SGLang for Kimi K3 NVFP4: this post covers what we changed in the attention, MoE/communication and KDA kernels, and how much each change bought. On the training side, Miles, which uses SG

   LMSYS Blog · 🔥 0

Experience and discussion from the community

Share my Megatron experienceAsk about Megatron