EgoXperience paper reconstructs camera, scene, hands and objects from egocentric video
An arXiv paper posted on 19 October, "EgoXperience: Learning World-Grounded Egocentric Representations from Human Experience," presents a full-stack pipeline spanning data collection, automatic annotation, unified modeling and benchmarking for reconstructing camera motion, scene geometry and hand-object interaction from egocentric video in one consistent world. At its center is EgoX-Engine, an annotation engine that couples geometrically verified object tracking with SLAM-based world alignment, producing EgoX-Data-6M, a 50-hour dataset of task-driven manipulation and locomotion, from which one hour is selected as the EgoX-Bench benchmark. The authors say their EgoX model takes monocular egocentric video plus first-frame object masks and, in a single forward pass, jointly reconstructs camera, scene, hands and objects in a shared metric world frame; against the strongest evaluated baselines it cuts camera, hand and depth errors by 68.9%, 65.6% and 42.9% and improves object pose scores by 28.0%.
The authors argue existing methods and datasets offer limited support for joint reconstruction and aligned supervision, which EgoXperience tries to unify through its annotation engine.
Computer Science > Computer Vision and Pattern Recognition
arXiv:2610.26625 (cs)
[Submitted on 19 Oct 2026]
Title:EgoXperience: Learning World-Grounded Egocentric Representations from Human Experience
Authors:Xinhao Ji, Hui En Pang, Hao Li, Kairui Hu, Jiner Wang, Mingze Sun, Ziang Cao, Yalun Dai, Thang Luu, Qin Ma, Hongyi Chen, Yukang Cao, Fangzhou Hong, Zhaoxi Chen, Ziwei Liu
View a PDF of the paper titled EgoXperience: Learning World-Grounded Egocentric Representations from Human Experience, by Xinhao Ji and 14 other authors
#EgoXperience#EgoX#EgoX-Engine#EgoX-Data-6M#EgoX-Bench#arXiv