BaGuaLu-2

Headlines · 1

1. Oct 28
   BaGuaLu-2 brings runtime-adaptive activation management to MoE training

   机器之心 reports that a Tsinghua University-led team’s BaGuaLu-2 paper was accepted by SC26. The system uses runtime load information to decide separately for individual operators whether to save, recompute, or offload activations, while scheduling additional work around pipeline gaps. The team released code for its activation-memory analysis model and scheduler-integrated offloading and reload mechanisms, along with integration examples and a neutral evaluation benchmark; the public repository does not include every runtime mechanism described in the paper. In team-reported tests on 128 to 12,288 Hygon DCUs with models from 188 billion to 1.04 trillion parameters, the system achieved average throughput gains of 1.14× over Megatron-Kuai and 1.13× over AdaPipe.

   机器之心 · 🔥 0

Experience and discussion from the community

Share my BaGuaLu-2 experienceAsk about BaGuaLu-2