ALPS visual state encoder unifies state probing, language and pixel reconstruction
A team of eight authors posted ALPS on arXiv on 21 October 2026, describing a pretrained visual state encoder for world models that aims to combine physical-state information, native language alignment and pixel reconstruction in a single set of patch tokens. ALPS is a ViT-B/16 pretrained from scratch on still images and text with language, reconstruction and spatial-correspondence objectives, then refined through cross-view self-distillation. In the authors' unified evaluation across fifteen public encoders, ALPS reports the lowest aggregate LIBERO state-probe error among ViT-B models, matching the 3.5x larger V-JEPA 2-L, and frozen ALPS tokens reportedly reach 85.5% mean policy success on LIBERO with the authors' native text tower; those benchmark results are author-run and have not been independently replicated.
The paper argues that a world model's visual foundation should support prediction, instruction and inspection at once, and reports that a single ViT-B encoder can match a much larger model on state probing.
Computer Science > Computer Vision and Pattern Recognition
arXiv:2610.30120 (cs)
[Submitted on 21 Oct 2026]
Title:ALPS: Visual State Encoders for World Models
Authors:Yanqing Liu, Letian Zhang, Chengzhi Liu, Zijun Wang, Jieru Mei, Hongliang Fei, Yuyin Zhou, Cihang Xie
View a PDF of the paper titled ALPS: Visual State Encoders for World Models, by Yanqing Liu and 7 other authors