Multi-Teacher On-Policy Distillation
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 3
- WAM-Policy brings language-conditioned control to quadrupedal robots
The authors of an arXiv paper present WAM-Policy, a two-stage method that adapts a world action model pretrained for robot arms to language-conditioned locomotion and manipulation on arm-free quadrupeds. Their method combines cross-morphology adaptation, on-policy self-distillation, reinforcement learning and multi-teacher distillation; on the authors' QLM-Bench, an OPSD student retained a 76.7% score while sampling 14 times faster, and later training raised it to 85.0%. The authors also report deploying the OPSD student on a real Unitree Go2 Air, although these results currently come solely from their paper and abstract.
Hugging Face · Papers · 🔥 0 - M3OPD-Reasoner opens multimodal post-training recipe
The M3OPD-Reasoner paper presents an open recipe for post-training vision-language models across image STEM, image perception, and video reasoning. Starting from Qwen3.5-9B, the authors report that multi-teacher on-policy distillation produced a 9B student closely matching each teacher in its domain and outperforming mixed-training baselines on average across 21 benchmarks; they also released the datasets, training code, and stage checkpoints.
Hugging Face · Papers · 🔥 0 - On-policy distillation can fail to preserve generalization
A preprint by Wang and six other authors extends on-policy distillation with a multi-teacher approach for consolidating capabilities from several teachers. Using Qwen3 models across mathematical reasoning, coding, agent, and long-context tasks, the authors report that strong in-distribution results can mask failures outside the training distribution: extrapolative scores were 2.45%–16.25% below the base model, while compositional scores were 18.19%–24.34% below the best teachers. The authors propose geometry-based patching and base-preserving replay to improve out-of-distribution performance while largely preserving in-distribution performance.
Hugging Face · Papers · 🔥 0
Experience and discussion from the community
Share my Multi-Teacher On-Policy Distillation experienceAsk about Multi-Teacher On-Policy Distillation
Nobody has shared their experience with Multi-Teacher On-Policy Distillation yet.