Test-time regression powers a new sequence compressor
A technical report by Sihan Yang and six coauthors introduces TTR-Compressor, which represents a compressed sequence using coefficients fitted through test-time regression. The regression is trained end to end through backpropagation, while coefficients are calculated in closed form during the forward pass; in author-reported tests, the method improved RULER performance with block-wise sparse attention and matched a standard MLP merger with 55.6% fewer visual tokens.
TTR-Compressor offers a reported alternative to pooling and learned projections for reducing sequence length in language and multimodal models, although its benchmark gains remain unverified.
Computer Science > Machine Learning
arXiv:2610.31574 (cs)
[Submitted on 22 Oct 2026]
Title:Rethinking Sequence Compression with Test-time Regression
Authors:Sihan Yang, Jie Huang, Hanxuan Li, Yifan Guo, Weigao Sun, Yiran Zhong, Weiyang Liu
View a PDF of the paper titled Rethinking Sequence Compression with Test-time Regression, by Sihan Yang and 6 other authors