Researchers successfully train robots to perform surgery by watching videos

What just happened? In a significant leap forward in medical technology, researchers have developed autonomous surgical robots. However, there are many issues that need to be addressed before these machines are actually used on humans.

Researchers from Johns Hopkins University and Stanford University have successfully trained robots to perform surgical tasks with the precision of human doctors by watching videos. This advancement, presented at the recent Conference on Robot Learning in Munich, marks a significant step towards more autonomous surgical robots and could be a partial solution to the looming shortage of surgeons in the US.

Robotic assistance in surgery is not new. Since 1985, when the PUMA 560 first assisted in a brain biopsy, robots have been helping doctors perform various procedures, including gallbladder removals, hysterectomies, and prostate surgeries. These robots, guided by doctors using joystick-like controllers, have been instrumental in minimizing human hand tremors during delicate procedures.

However, the recent breakthrough takes this technology to a new level. The research team has developed robots capable of performing complex surgical tasks autonomously, including manipulating needles, tying knots, and suturing wounds. What sets these robots apart is their ability to learn from videos and correct their mistakes without human intervention.

The team’s approach to training these robots is similar to that used in developing language models like ChatGPT. However, instead of working with words, the system employs a language that describes the position and direction of the robot’s gripper.

“We built our training model using videotapes of robots performing surgical tasks on practice suture pads,” Dr. Axel Krieger, an associate professor at Johns Hopkins Whiting School of Engineering who supervised the research, explained to The Washington Post. “Each image in the video sequence is converted into numerical data, which the model then translates into robot actions.”

This method significantly reduces the need for programming each individual movement required for a medical procedure. The trained robots demonstrated their skills in a different environment, successfully performing tasks on pork and chicken samples. “We’ve developed a system where you can talk to the robot like you would to a surgical resident,” Ji Woong “Brian” Kim, a postdoctoral researcher on the team, said. “You can say things like, ‘Do this task,’ or ‘Move left’ and ‘Move right.'”

The development of more autonomous surgical robots could help address the projected shortage of 10,000 to 20,000 surgeons in the United States by 2036, according to the American Association of Medical Colleges. “We’re not trying to replace the surgeon. We just want to make things easier for the surgeon,” Dr. Krieger said.

While the progress is impressive, experts say numerous challenges remain before fully autonomous surgical robots become a reality. “The stakes are so high because this is a life and death issue,” Dr. Dipen J. Parekh, director of robotic surgery at the University of Miami Miller School of Medicine, said. “The anatomy of every patient differs, as does the way a disease behaves in patients.”

Furthermore, as the technology advances, it raises important questions about responsibility, privacy, and access. Dr. Amer Zureikat, director of robotic surgery at the University of Pittsburgh Medical Center, noted several concerns about accountability in the event of surgical errors. Determining liability when multiple parties are involved in the development and use of autonomous surgical robots would be complex, to say the least, with potential culpability extending to various stakeholders, including the supervising physician, the AI developers, the hospital administration, or even the robot manufacturers themselves.

Privacy concerns also loom large, particularly regarding the use of real surgical videos for training these systems. Additionally, there are questions about equal access to the technology and the potential for surgeons to become overly reliant on robotic assistance.