LoopVQ Targets Sub-1-Bit KV Cache Quantization
Minsoo Cheong and Sungjoo Yoo’s paper introduces LoopVQ, a KV cache vector quantization method for looped transformers that jointly codes channels across multiple loop steps and fine-tunes codebooks against full-precision outputs. The authors report that it outperforms existing vector quantization methods on general, long chain-of-thought reasoning and long-context retrieval tasks with Ouro and Huginn at 0.5 bits per element. In SGLang, the implementation nearly halves decode latency at a 32K-token context and raises peak serving throughput by up to 4.3× and 6.1× for the 1.4B and 2.6B Ouro models versus BF16.
LoopVQ directly addresses the linear KV-cache growth that makes repeated loop steps a memory bottleneck, and the authors report an implemented SGLang integration with lower latency and higher serving throughput.
Computer Science > Machine Learning
arXiv:2611.05304 (cs)
[Submitted on 3 Nov 2026]
Title:LoopVQ: Sub 1-bit KV Cache Quantization for Looped Transformers
Authors:Minsoo Cheong, Sungjoo Yoo
View a PDF of the paper titled LoopVQ: Sub 1-bit KV Cache Quantization for Looped Transformers, by Minsoo Cheong and 1 other authors