TraceForge distills teacher answers into student-style reasoning traces
A single-author arXiv preprint submitted on 21 October 2026, "TraceForge: Distilling with Your Own Thinking" by Yan Wang, proposes generating a reasoning trace from a teacher model's context and answer in the student model family's own style, instead of recovering hidden reasoning by replaying encrypted blocks — an approach the author says breaks providers' terms of service. Training is two-stage, supervised then adversarial, with a generator writing traces and a critic judging how real a reasoning trace looks. The author reports that an independent judge scored generated traces at 0.68 average authenticity versus 0.66 for real traces and 0.82 for critic-selected traces from four candidates, and that with the same 5,000 questions and teacher answers HumanEval+ improved 3.7 points and MBPP+ 7.1 at the final checkpoint; the paper notes these are authenticity scores, not correctness rates, and the results have not been independently verified.
The paper proposes a route to distillation data that does not rely on replaying encrypted reasoning blocks, which the author argues violates providers' terms of service.
Computer Science > Artificial Intelligence
arXiv:2610.29680 (cs)
[Submitted on 21 Oct 2026]
Title:TraceForge: Distilling with Your Own Thinking
Authors:Yan Wang
View a PDF of the paper titled TraceForge: Distilling with Your Own Thinking, by Yan Wang