Fin-SD研究：通用反馈媲美详细批评
Fin-SD 研究人员报告了面向 GPT-OSS-20B 多步金融推理的事后自蒸馏方法：学生模型先找出首个错误步骤并生成批评，再由冻结副本在反向 KL 目标下将结果蒸馏回学生模型。作者关联的 Reddit 摘要称，固定通用反馈可达到自写详细批评约 3 分的同等增益，而大 6 倍模型的批评未带来显著提升；Fin-SD 还以少 9.5% 的 token 数量追平使用完整参考解答训练的 OPSD。
报告结果表明，事后蒸馏提供的位置信息可能已能带来大部分训练收益，而无需依赖详细批评或规模显著更大的模型反馈。
We study hindsight self-distillation for multi-step financial reasoning on GPT-OSS-20B. The student finds its own first wrong step and writes a critique. A frozen copy, conditioned on the prefix plus the critique, is distilled back into the student under reverse KL. Notably, replacing the critique with a fixed generic feedback line matches the full gain of about 3 points, and critiques from a 6x larger model do not materially improve performance, suggesting that the positional information provided by hindsight distillation significantly decreases the privileged information needed for effective training. Fin-SD matches OPSD from full reference solutions with 9.5% fewer tokens.
Paper: https://arxiv.org/abs/2610.36082 | Code: https://github.com/johnnyhyu/fin-sd
We'd love to find the limits of this approach. Let us know your thoughts!
submitted by /u/jhzyu [link] [comments]