veRL

Headlines · 3

1. Oct 20
   AWS fine-tunes GPT-OSS-20B for multilingual reasoning

   AWS Machine Learning Blog reports that supervised fine-tuning raised GPT-OSS-20B’s reasoning-language accuracy from 22% to 100% on a 50-case test covering English, French, German, Spanish and Italian. Applying GRPO after SFT increased answer-language accuracy from 62% to 70%, while a Qwen2.5-7B control experiment added 6 points on multilingual reasoning but produced no gain on GSM8K.

   AWS Machine Learning Blog · 🔥 0
2. Oct 14
   vLLM open-sources bitwise-consistent RL training kernels and an agent skill

   A vLLM blog post argues that inference-side optimizations in high-performance RL — low-precision compute, low-precision KV cache, kernel and communication tuning — shift the numerics, so the training engine assigns different probabilities to the same token than the engine that sampled it; on DeepSeek-V4-Flash the high-performance baseline's train-inference KL rose from 2e-3 to 29.9 as reward fell back. The proposed fix has the training forward call vLLM's inference kernels directly, modified so rollout and training produce bitwise-identical logprobs for every sampled token (diff = 0). The authors report diff of 0 at every one of 100 DeepSeek-V4-Flash steps at 91.3% of the baseline's rollout throughput, and say that on Nemotron-3.5 (a Mamba hybrid with NVFP4) the consistent run's SWE-bench Pro score held near 0.6 while the baseline's fell from 0.56 to 0.22. The workflow is packaged as an agent skill and released with recipes and code, and the team says DeepSeek-V4.1 has reached 0 diff with performance work still in progress.

   vLLM Blog · 🔥 0
3. Oct 10
   Xiaomi's MiMo-V2.6 report details $2.6M single-run RL post-training

   Xiaomi's MiMo team published a technical report, "MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement," covering two sparse MoE models: MiMo-V2.6-Pro (1.02T total parameters, 42B activated) and MiMo-V2.6-Flash (310B total, 15B activated). The report says each RL step samples 1,568 prompts and roughly 25,000 agent trajectories totalling 2.7–3.7B tokens, with RL post-training costs of about $2.6M for Pro and $900K for Flash. The team also open-sourced more than 7,000 RL environments plus an end-to-end RL framework.

   InfoQ 中文 AI&大模型 · 🔥 0

Experience and discussion from the community

Share my veRL experienceAsk about veRL