Megatron-MUSA-Titan

Headlines · 1

1. 4d
   Moore Threads Open-Sources Megatron-MUSA-Titan for Large-Scale MoE Training

   Moore Threads announced the open-sourcing of Megatron-MUSA-Titan, a large-model training infrastructure project adapted for the company’s MUSA platform and full-function GPUs. IT之家 reported that it targets trillion-parameter MoE pretraining and reinforcement learning on clusters with 10,000 GPUs. The project supports blockwise FP8 training and can work with the DeepEP and ACE asynchronous communication engines to overlap expert parallelism and computation. Moore Threads said the project had been used in continued trillion-parameter MoE pretraining on its 10,000-GPU-scale cluster and supports coordination with slime and miles, Megatron training, and SGLang sample generation. Its reported model FLOPS utilization and long-running stability were not independently verified in the supplied reports.

   IT之家 AI · 🔥 0

Experience and discussion from the community

Share my Megatron-MUSA-Titan experienceAsk about Megatron-MUSA-Titan