Ask-to-Distill

Headlines · 1

1. Oct 19
   Ask-to-Distill optimizes synthetic questions for on-policy distillation

   A new arXiv preprint (2610.27450, submitted 20 Oct 2026) by Jiashuo Sun and 11 co-authors introduces Ask-to-Distill, a framework that treats synthetic questions as "distribution elicitors" and optimizes the teacher supervision they expose along student-generated trajectories. A student-initialized question synthesizer is trained using a distributional surrogate for question value computed from a frozen student–teacher pair, without reference answers, correctness rewards or per-candidate student updates; the reward combines teacher–student disagreement with teacher probability mass on the student's candidate support, plus length and diversity controls. The authors report that across mathematical reasoning, code generation and agentic tasks, the method improves average performance and teacher-gap recovery over unoptimized student-generated prompts and KL-filtered real questions, and matches or exceeds real-bank on-policy distillation in the evaluated settings, with code, data and models stated as publicly available.

   Hugging Face · Papers · 🔥 0

Experience and discussion from the community

Share my Ask-to-Distill experienceAsk about Ask-to-Distill