Think-Graft folds an early-exit probe into reasoning model weights
A Reddit author says Think-Graft requires no model training or extra inference-time code, instead folding a linear probe into the output-layer weight row for the end-of-thinking token. In the author’s small tests, Qwen3.5-4B used 54% fewer tokens with unchanged accuracy on eight unseen MATH-500 problems, while DeepSeek V4.1 Flash used 40% fewer tokens and solved 7 of 7 maths problems compared with 6 of 7 for the original. In a coding-agent test building a small website, the author reports 84% less thinking and says page quality held up. The author notes that the results use small samples, the probe was fitted only on maths problems, and the AIME 2025 comparison used a 32,000-token limit rather than the 81,920 tokens recommended for Qwen.
The method offers a weight-only way to make reasoning models stop repeated checking without training or adding a runtime component, but its reported benefits come from small author-run samples and transfer beyond maths remains uncertain.
Reasoning models often reach the right answer early and then keep double-checking it for thousands of tokens. I got the model to stop at that point by changing a single row of weights..There's no training and no extra code at inference time; the edited model is an ordinary weights file that runs anywhere.
Processing img 5yk991rjlt1i1...
Background,
- Reasoning models (DeepSeek, Qwen, and so on) "think" in a scratchpad before answering. The scratchpad ends when the model writes a special token, </think>.
- To pick each next token, the model first computes a summary of everything so far: a long list of numbers called the hidden state. The last layer, the output layer, has one row of weights per word in the vocabulary. Each word's score is its row multiplied by the hidden state, and the highest-scoring words are the likeliest next words.
- So the score for "stop thinking now" (</think>) comes from exactly one row of that layer.
What I did
- Ran the model on 24 maths problems and recorded its hidden state at every token of its thinking.
- Marked the points where the right answer had already appeared and the model was only re-checking (specifically, the ends of paragraphs after that point), and treated everything else as "still working".
- Found the direction in the hidden state that best separates those two groups. This is a probe: a simple linear classifier (linear discriminant analysis), computed directly with linear algebra, with no gradient training.
- Added that direction, times a strength of 14, to the </think> row. The "stop" score now rises only when the model's state looks like "done, just re-checking". While it's still solving, nothing measurable changes.
#Think-Graft#Qwen3.5-4B#DeepSeek V4.1 Flash#llama.cpp#MATH-500#AIME 2025