Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Study finds bias in on-policy self-distillation

AI summary

A research update argues that on-policy self-distillation (OPSD) can bias a model toward a teacher’s preferred continuations, including stylistic tokens such as hedging words, without ensuring improvement in its ability to solve tasks. Token-level analysis covering 800 rollouts across 200 level 4–5 MATH-500 problems found systematic disagreements over wording rather than mathematical correctness. The authors conclude that OPSD is not a silver bullet for continual learning and question whether further research investment is worthwhile, while acknowledging that it may remain useful in certain contexts.

In-depth insight

The token-level findings challenge the assumption that dense teacher signals from OPSD reliably indicate better reasoning, because much of the observed signal concerns stylistic differences rather than mathematical correctness.

Foundations

The bias of on-policy self-distillation

Why on-policy self-distillation is not the silver bullet for continual learning.

Authors

Date

November 6, 2026

#Baseten#On-policy self-distillation#Qwen3-8B#MATH-500

Original · 13 min readRead the original
Group chat

No comments yet. Start the conversation.