VPP2 and Dyna-2: how human video trains robots
A TMTPost feature by Jushenzhi reports that on October 10 Robot Era open-sourced two sets of VPP2 video model weights; VPP2 splits manipulation learning into predicting how a task unfolds, compressing it into visual plans and learning robot actions, and only the first two stages were released. Dyna Robotics' Dyna-2 is described as a scale test: four models pretrained on 1,000, 10,000, 100,000 and 1 million hours of first-person human video then received identical robot-task training, the first three failed to turn a key while the 1-million-hour model succeeded 9 of 10 times. Nvidia's DreamZero added 12 minutes of action-label-free human video for nine previously unseen tasks and lifted average task progress from 38.3% to 54.3%, while RoboWM-Bench from Peking University and others converted generated video into robot actions in simulation, reporting 50%, 20% and 0% success for Wan 2.6 on closing drawers, picking objects and placing objects. The article argues human video is taking over task understanding and world dynamics while scarce robot data focuses on body control.
If human video can substitute for some real-robot demonstrations, scaling robot learning would no longer depend entirely on collecting data for every body and task.