ThinkBit proposes two-bit quantization for reasoning models
Rhea Sanjay Sukthanker and Pashmina Cameron submitted the ThinkBit paper to arXiv on October 26, 2026. The framework uses layer-specific bit allocation, Gumbel-softmax-based scalar quantization, scale recalibration and quantization-aware training to preserve reasoning while compressing weights to two bits. The authors report that a 0.52 GB Qwen3-1.7B checkpoint scored 46.7 on MATH-500 after recalibration and 73.0 after QAT, while a 3.74 GB model scaled to 14B scored 94.8 versus 97.5 for the 29.5 GB bfloat16 model; these results remain author claims from a single paper.
The authors say that reasoning-model generation is limited by memory bandwidth and that storing weights in two bits instead of bfloat16 could make inference up to 8 times cheaper. The reported framework and results address the challenge of reducing model storage while preserving reasoning performance.
Computer Science > Artificial Intelligence
arXiv:2610.34886 (cs)
[Submitted on 26 Oct 2026]
Title:ThinkBit: Search Codewords, Recalibrate Scales, Select by Reasoning
Authors:Rhea Sanjay Sukthanker, Pashmina Cameron
View a PDF of the paper titled ThinkBit: Search Codewords, Recalibrate Scales, Select by Reasoning, by Rhea Sanjay Sukthanker and 1 other authors