Nvidia Pioneers a New Era in Healthcare Robotics with Physical AI
Nvidia is making groundbreaking strides in the world of healthcare robotics by introducing a new medical physics simulation framework. This innovative approach treats healthcare robots as physical AI systems, which require an embodied experience to learn effectively, rather than relying solely on code. This development is poised to revolutionize the way surgical and diagnostic robots are trained and deployed in medical settings.
Understanding the Concept of Physical AI
The term “Physical AI” is increasingly being adopted by Nvidia and other players in the robotics industry to describe machines that learn about the world through direct interaction. Unlike traditional AI models that learn from text or images, physical AI systems gain knowledge from physical experiences, such as the force of a catheter hitting a vessel wall or the pressure exerted by a robotic arm on soft tissue. This learning typically requires a physical body acting in the real world or a detailed simulation to act as a substitute.
In the field of healthcare robotics, the physical bodies required for real-world procedures are not only scarce but also subject to strict regulations. This scarcity limits the range of scenarios that a robot needs to experience to be effective. Nvidia’s Medical Physics Simulation aims to computationally replicate this embodied experience, providing a solution to this challenge.
Building Physical Intuition Before Using a Scalpel
Nvidia’s framework, introduced as an open-source addition to its Isaac for Healthcare platform, simulates the physical interactions that would take years for a surgical or diagnostic robot to encounter in clinical use. These scenarios include borderline cases such as a guidewire caught on a calcified vessel wall, a kidney stone lodged at an unusual angle, or rare soft tissue reactions. By generating these scenarios on demand, developers can train robots to handle a wide array of situations before they ever enter an operating room.
The framework combines two approaches to model device behavior inside a body. Classical physics simulation processes the known mechanical rules, such as how a catheter bends or the resistance exerted by a vessel wall. Generative AI, through a component called Cosmos-H Dreams, handles the more complex aspects like visual scene dynamics, learned from procedural data. This combination allows robots to understand both the physics and the visual and anatomical variations they might encounter.
Nvidia’s use of GPUs and its Warp and Newton libraries enables the framework to run numerous parallel training environments simultaneously. In a benchmark test, 8,192 parallel environments reduced training time from over five hours to under two minutes. However, while this demonstrates impressive throughput, it does not guarantee clinical reliability. The challenge remains to ensure that these simulated failure modes correspond to actual failures in a real-world operating room.
Testing the Embodiment Approach
Several organizations are early adopters of Nvidia’s physical AI approach, applying it to different degrees. CMR Surgical and Cambridge Consultants, owned by Capgemini, have contributed approximately 500 hours of anonymized clinical data from the Versius Surgical Robotic System to the Open-H Embodiment dataset. This includes data from various procedures and uses Cosmos-H Dreams to model soft tissue interactions and create patient-specific simulations.
Johnson & Johnson MedTech is using the framework to develop a digital twin of its endoluminal MONARCH platform, focusing on kidney stone scenarios in urology. XCath applies the framework for endovascular autonomy training, while Inner Logic generates synthetic data to validate device mechanics. Medtronic Structural Heart is exploring simulated X-ray sensing for catheter navigation research.
While these efforts represent training exercises or contributions to datasets, none of these systems are yet deployed in clinical settings. Nvidia, however, is not claiming otherwise, highlighting the distinction between training and real-world deployment.
The Open Source Advantage in Physical AI Systems
In healthcare robotics, there is a significant regulatory requirement for transparency in how AI systems arrive at their behaviors. An open-source framework allows for the validation of physical assumptions within simulations, reproduction of results across different anatomies, and creation of an evidence path suitable for regulatory submissions to bodies like the FDA.
Open code enables third-party reviewers to verify a model’s logic, contributing to the trustworthiness and reliability of the system. However, it does not replace the need for thorough testing to confirm that a model’s physical behavior matches real-world scenarios, a validation that has not yet been published by companies using Nvidia’s framework.
Nvidia’s infrastructure shortens the pre-hardware phase of physical AI development for surgical and diagnostic robots. The ability to conduct training at scale in parallel is a shift from creating a custom simulation scene for each workflow, marking a significant advancement in healthcare robotics.
For more information on this topic, visit the Physical AI Expo.
See also: Bristol Myers Squibb buys Nvidia AI system for drug discovery
Want to learn more about AI and big data from industry leaders? Check out the AI & Big Data Expo taking place in Amsterdam, California and London. The comprehensive event is part of TechEx and takes place alongside other leading technology events including the Cyber Security & Cloud Expo. Click here for more information.
AI News is operated by TechForge Media. Discover more upcoming enterprise technology events and webinars here.
“`
