NVIDIA releases Cosmos3-H-Dreams for generative surgical simulation
NVIDIA introduced Cosmos3-H-Dreams, a research simulator that generates interactive cholecystectomy scenes from video and robot kinematics recorded on the CMR Surgical Versius platform. Built on the 16-billion-parameter Cosmos3-H-Embodied-Nano model, it includes separate four-step and two-step student checkpoints for real-time generation with FlashDreams on one NVIDIA RTX PRO 6000 GPU. NVIDIA says the system is intended for research and is not a validated clinical tool.
The release provides researchers with an open route to study action-conditioned, generative simulation of clinical surgical scenes, although its anatomical accuracy and clinical validity remain unvalidated.
Picture moving the Cadiere grasper, grasping the gallbladder, and lifting it, all within a generated surgical scene that responds in real time. Today, we take the next step and introduce Cosmos3-H-Dreams, bringing interactive generative simulation to the visual and procedural complexity of clinical surgery with our most convincing in-body imagery to date.
Our first Cosmos-H-Dreams release, built on Cosmos-Predict2.5, brought interactive generative simulation to dVRK tabletop suturing, a foundational proving ground for surgical robotics. As world foundation model (WFM), Cosmos-H-Dreams demonstrated an alternative to numerical simulation: rather than explicitly modeling scene geometry, tissue deformation, and physical interactions, it learned visual dynamics directly from synchronized video and robot kinematics, yielding interactive tabletop task simulation with high realism.
Today, Cosmos3-H-Dreams advances this capability by learning from video and robot kinematics recording during clinical procedures on the CMR Surgical Versius platform. To our knowledge, this is the first real-time generative surgical simulator trained on clinical cholecystectomy recordings to support live kinematic control. Powered by Cosmos 3 and accelerated with FlashDreams, NVIDIA’s accelerated streaming-inference library, Cosmos3-H-Dreams runs interactively on a single NVIDIA RTX PRO 6000 GPU.
#NVIDIA#Cosmos3-H-Dreams#Cosmos 3#Cosmos3-H-Embodied-Nano#FlashDreams#Open-H-Embodiment#CMR Surgical Versius#NVIDIA RTX PRO 6000