Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

VPP2 and Dyna-2: how human video trains robots

AI summary

A TMTPost feature by Jushenzhi reports that on October 10 Robot Era open-sourced two sets of VPP2 video model weights; VPP2 splits manipulation learning into predicting how a task unfolds, compressing it into visual plans and learning robot actions, and only the first two stages were released. Dyna Robotics' Dyna-2 is described as a scale test: four models pretrained on 1,000, 10,000, 100,000 and 1 million hours of first-person human video then received identical robot-task training, the first three failed to turn a key while the 1-million-hour model succeeded 9 of 10 times. Nvidia's DreamZero added 12 minutes of action-label-free human video for nine previously unseen tasks and lifted average task progress from 38.3% to 54.3%, while RoboWM-Bench from Peking University and others converted generated video into robot actions in simulation, reporting 50%, 20% and 0% success for Wan 2.6 on closing drawers, picking objects and placing objects. The article argues human video is taking over task understanding and world dynamics while scarce robot data focuses on body control.

In-depth insight

If human video can substitute for some real-robot demonstrations, scaling robot learning would no longer depend entirely on collecting data for every body and task.

Robot EraVPP2Dyna Robotics

Original · 10 min readRead the original
Group chat

No comments yet. Start the conversation.