Megatron-LM

Headlines · 2

1. 6d
   vLLM open-sources bitwise-consistent RL training kernels and agent skill

   vLLM published a method that makes RL training and inference bitwise consistent: the training forward calls vLLM's inference kernels directly, and those high-performance kernels are modified so rollout and training produce identical logprobs for every sampled token. The team reports that on DeepSeek-V4-Flash its high-performance baseline's train-inference KL rose from 2e-3 to 29.9 while reward fell back, whereas the consistent run kept diff at 0 for 100 steps at 91.3% of the baseline's rollout throughput. On Nemotron-3.5, a Mamba hybrid with NVFP4, the consistent run's SWE-bench Pro score held around 0.6 while the baseline dropped from 0.56 to 0.22; the workflow is released as an agent skill with recipes and code, and DeepSeek-V4.1 has reached 0 diff with performance work still in progress.

   vLLM Blog · 🔥 0
2. 6d
   Modal tests mixed-precision, mixed-accelerator RL samplers on Qwen3.6-35B-A3B

   Modal published an engineering post asking whether large-model reinforcement learning can use samplers that run different precisions and accelerators than the trainer, each with its own kernels. Its mixed samplers span six accelerator types — B200 and B300 serving NVFP4 weights, H100 and H200 serving FP8 and BF16, and A100 and RTX PRO 6000 serving BF16 — all with FP8 KV caches, feeding a single BF16 trainer on B200. Modal says off-policy correction plus autoscaling serverless GPUs let it train Qwen3.6-35B-A3B fully asynchronously on agentic coding tasks with up to 256K-token contexts, reporting that the mixed samplers' mismatch averaged 4.7× that of homogeneous samplers.

   Modal Blog · 🔥 0

Experience and discussion from the community

Share my Megatron-LM experienceAsk about Megatron-LM