Apple's LensVLM compresses long context into images, then zooms in
Apple's foundation-model team and Duke University propose LensVLM, which first lets a model skim compressed page images that cover the full text, then calls a tool to expand the pages relevant to the question and read the original content. The paper reports 68.9% average accuracy across seven text QA benchmarks at 4.3x effective compression, close to the 72.4% full-text reference; under 15x initial compression the base model scores 18.1% versus 52.1% for LensVLM. Built on Qwen3.5-9B-Base, it uses Qwen3.5-397B to generate tool-use trajectories for supervised fine-tuning, followed by DAPO reinforcement learning; the work was accepted to NeurIPS 2026 and a LensVLM 9B demo is online.
As context windows keep growing, teaching models to allocate their own reading budget — what to skim and what to zoom into — may matter as much as raw window size.