V-CoLA: training-free vision token compression for linear-attention VLMs
V-CoLA is a training-free vision token compression framework designed for hybrid architectures that use linear attention (the paper cites Qwen3.5 as an example). It introduces a uniqueness-aware importance criterion to pick critical vision tokens and an adaptive token merging strategy to compress them, with all components tuned to stay compatible with the chunk-wise parallelism of linear attention. The authors report reaching 99.5% of original performance with only 50% of vision tokens, over 88% with as few as 12.5%, and 1.86x to 6.15x prefill speedups across multiple benchmarks. They also note that prior softmax-attention-based attention or similarity compression methods degrade noticeably in this regime.
If reproducible, this could meaningfully cut inference cost for linear-attention vision-language models working with long visual contexts.
Computer Science > Computer Vision and Pattern Recognition
arXiv:2610.11251 (cs)
[Submitted on 8 Oct 2026]
Title:V-CoLA: Vision Token Compression with Linear Attention
Authors:Hao Jiang, Yiru Mao, Tianpeng Bu, Hao Zhou, Hongtao Duan, Wang Jing, Bowen Xu, Xin Chen, Lulu Hu, Bin Yang, Yongliang Tao, Minying Zhang
View a PDF of the paper titled V-CoLA: Vision Token Compression with Linear Attention, by Hao Jiang and 11 other authors