RobotWorld benchmark tests multimodal agents on 84 physical robot tasks
A multi-institution team released RobotWorld, a simulation testbed for “robot use” in which multimodal agents turn instructions and observations into physical task execution; its 84 tasks span manipulation, mobile manipulation, locomotion, driving and aerial control, with explicit interaction budgets and executable success checks. The paper (62 pages, 25 figures) finds that agents can build sophisticated perception and control workflows — image segmentation, camera calibration, spatial estimation and dynamics-based computation — yet these do not consistently compose into success: agents lose task-relevant object states after reaching commanded poses, fail to correct ineffective actions, recover too late, or mistake unfinished tasks for completion. Performance differs by model, with Astra succeeding more often on spatial and constrained-contact goals and Opus 5.5 on continuous-balance and timed-interaction goals.
It uses execution traces to pinpoint where digital agent skills fail to transfer to physical robots, giving concrete targets for training more reliable embodied agents.