创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

Think-Graft 将提前退出探针折叠进推理模型权重

AI 总结

一名 Reddit 作者称,Think-Graft 无需重新训练模型,也无需增加推理时代码,而是把线性探针折叠进结束思考标记对应的输出层权重行。在作者进行的小规模测试中,Qwen3.5-4B 在 8 道未见 MATH-500 题上准确率不变,Token 减少 54%;DeepSeek V4.1 Flash 的 Token 减少 40%,数学题成功数为 7/7,而原模型为 6/7。在一个构建小型网站的代码智能体测试中,作者报告思考量减少 84%,并称页面质量保持不变。作者指出,这些结果样本量较小,探针仅使用数学题拟合,且 AIME 2025 对比采用了 32,000 Token 上限,低于 Qwen 针对竞赛数学推荐的 81,920 Token。

深度见解

该方法提供了一种仅修改权重、无需训练或增加运行组件来减少重复推理的路径,但报告的收益主要来自作者自测的小样本,方法能否迁移到数学以外的领域仍不确定。

Reasoning models often reach the right answer early and then keep double-checking it for thousands of tokens. I got the model to stop at that point by changing a single row of weights..There's no training and no extra code at inference time; the edited model is an ordinary weights file that runs anywhere.

Processing img 5yk991rjlt1i1...

Background,

  • Reasoning models (DeepSeek, Qwen, and so on) "think" in a scratchpad before answering. The scratchpad ends when the model writes a special token, </think>.
  • To pick each next token, the model first computes a summary of everything so far: a long list of numbers called the hidden state. The last layer, the output layer, has one row of weights per word in the vocabulary. Each word's score is its row multiplied by the hidden state, and the highest-scoring words are the likeliest next words.
  • So the score for "stop thinking now" (</think>) comes from exactly one row of that layer.

What I did

  • Ran the model on 24 maths problems and recorded its hidden state at every token of its thinking.
  • Marked the points where the right answer had already appeared and the model was only re-checking (specifically, the ends of paragraphs after that point), and treated everything else as "still working".
  • Found the direction in the hidden state that best separates those two groups. This is a probe: a simple linear classifier (linear discriminant analysis), computed directly with linear algebra, with no gradient training.
  • Added that direction, times a strength of 14, to the </think> row. The "stop" score now rises only when the model's state looks like "done, just re-checking". While it's still solving, nothing measurable changes.

#Think-Graft#Qwen3.5-4B#DeepSeek V4.1 Flash#llama.cpp#MATH-500#AIME 2025

原文约 3 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。