NVIDIA introduces Cosmos3-H-Dreams, a real-time generative simulator for robot-assisted cholecystectomy

NVIDIA introduced Cosmos3-H-Dreams, a real-time generative surgical simulator trained on clinical cholecystectomy video and robot kinematics from the CMR Surgical Versius platform. Built on Cosmos 3 and accelerated with FlashDreams, it runs interactively on a single NVIDIA RTX PRO 6000 GPU and includes four-step and two-step checkpoints that trade visual quality for faster inference. The model uses 832 x 480 video with actions aligned at 10 Hz and supports 28 action dimensions for two instruments and related controls.

For surgical robotics teams, the reported recipe and checkpoints offer a path to adapt generative simulation to synchronized video-and-kinematics datasets, but controller integrations must match the training action conventions and normalization. The practical trade-off is latency vs. fidelity: the two-step model is faster, while the four-step model preserves more visual quality. A key limitation remains that NVIDIA describes visual realism as a starting point, not proof of biomechanical accuracy, with surgeon evaluation and stronger validation still listed as future work.