Could We Ever…Have Robotic Surgery Without the Surgeons?
Hosts
Ricardo Castrillón BA’17
Danyelle Jordan Gates BA’17
Audio Editor
Sarah Wall BA’19
Producers
Paul Bottoni
Brittany Magelssen
Katherine Morales
Phil Roth
Music by Roxanne Minnish MFA’11, senior lecturer in the UT Dallas School of Arts, Technology, and Emerging Communication
Artwork by Rachael Drury BA’19
The views expressed on this podcast by the hosts and guests do not reflect the views of The University of Texas at Dallas.
Show Transcript
[Ricardo] We’re recording? Okay. Welcome to Could We Ever, part of the….[mumbles] Okay. ever part of the column okay welcome to
[Danyelle] Welcome to Could We Ever, part of the UT Dallas CometCast Network.
[Ricardo] Could We Ever shines a light on our experts and ask them to tackle questions you never knew you needed answered.
[Danyelle] From science, to art and more.
[Will Smith] You are just a machine, an imitation of life. Can a robot write a symphony? Can a robot turn a canvas into a beautiful masterpiece?
[Robot] Can you?
[Ricardo] That was a clip from “iRobot” with Will Smith and as you might have guessed, we’re talking about robots.
[Danyelle] Today we’re talking to Dr. Ann Majewicz Fey. She’s an assistant professor of mechanical engineering in the Erik Jonsson School of Engineering and Computer Science, and she studies surgical robots. She’s gonna help us tackle the question of, “Could we ever have robots that do surgery without the surgeon?” Thanks for joining us today, Ann. First off, we want to get to know you. Tell us about your background in the area of surgical robots.
[Dr. Fey] I studied mechanical and electrical engineering in college because I was really interested in robotics, but at some point I wanted to do something that would ultimately help society in some way. So around that time I started looking to see, you know, what’s available in robotics and medicine and I just kind of stumbled upon this whole area of surgical robotics, which I found completely fascinating and that’s what I decided that I wanted to do for the rest of my life.
[Danyelle] And how did you initially get involved in robotics?
[Dr. Fey] So robotics, I started learning more about when I was an undergrad at the University of St. Thomas. I actually first was just a mechanical engineering student but they had this robotics student club through the electrical engineering student club and so I was really interested in learning more about what they were doing and they were using these sensors and programming the robot to do all of these crazy things that I never even learned about in my mechanical engineering training, and that’s actually why I decided to also get an electrical engineering degree.
[Ricardo] What an overachiever.
[Danyelle] Student organizations matter! Yes, yes!
[Dr. Fey] Yeah, student organizations make a difference.
[Danyelle] So cool.
[Ricardo] Anne actually had a really interesting story about how she got into engineering.
[Dr. Fey] I think growing up I used to always help my dad around at the house and and he was an engineer so I remember going with him to work and seeing all the different equipment that he was working on. He’s a manufacturing engineer. So that probably kind of got the wheels turning about engineering. And I actually remember he used to have this little competition between me and my sister on weekends for inventions and he would give us $20 if we had the better invention so maybe that’s kind of what started some of this interest in engineering for me. You know, when I, when I went through high school – middle school and high school – it was kind of before this whole wave now of doing engineering for, for girls and, and trying to do engineering kind of at younger levels, so I didn’t even know really what a robot was or I had no idea that you could program stuff until I actually got to college. Now I think there’s kids that program, you know, their LEGO robots and that’s something that didn’t really exist when I was young. So I’m not really sure how I got into it but, um, but I think just spending time you know working with my hands, helping my dad out, you know, fixing gardening equipment and stuff like that, fixing things in the house, I think all of that just kind of helped me realize that I wanted to build something with my career. Like I didn’t want to just think about things or talk about things. I actually wanted to do something and build things and I think that’s kind of how I got into it. Also in high school I was really interested in theater, so I actually was the only costume crew, the only person on costume crew for four years, in high school doing all the costumes for the plays that we had. And I think that, too, was a really formative experience because if you think about, you know, sewing and creating costumes, you’re really building something. You’re taking sort of this 2d fabric and you’re creating a really cool three-dimensional thing that someone’s gonna wear and move in and you have to think about how it’s gonna flow and I think having that kind of creative outlet when I was in high school really kind of helped me think about things in a more creative way when I then went on to become an engineer.
[Danyelle] And so what specifically interested you about healthcare and robotics?
[Dr. Fey] That’s a good question. I think what got me really interested in healthcare for robotics is just wanting to do something that I knew could make people’s lives better. There wasn’t really a specific incident or, you know, situation that made me interested in in health care in general, but I think just wanting to use my engineering talents to help people, that was sort of what, what drove me down this, down this path
[Danyelle] All right, so, we wanted to know how long surgical robots have been around and what do they do now?
[Dr. Fey] The first documented in case of surgical robotics actually happened in 1985 and it was a neurosurgical procedure. So essentially they use this commercial robot arm called the Puma to help a surgeon do a brain biopsy with a needle. So that was the first case. And since then it’s become a really active area not just for industry – so companies that want to design and build custom surgical robots for specific procedures – but it’s also been a really active area of research and that’s the area that I’m in now.
[Ricardo] So what, in terms of like, what its purposes are, like what problem did it solve introducing robots into the surgery?
[Dr. Fey] Initially the problem that robotics solves, in terms of medical procedures and interventional procedures, is having a device that can be repeatable and more consistent. So that’s what robots are good at. They’re good at doing basically the same kind of tasks over and over again in a very controlled environment. So initially when they started having these robots for needle insertion or for orthopedic bone surgery applications, they wanted to have a robot that they could program to do something very carefully and very stably in a way that maybe a human couldn’t do.
[Ricardo] In that case if robots are good at performing these tasks, why not just have the robots do it and not have a surgeon be involved in it?
[Dr. Fey] The reason why you might actually want to have a surgeon in the loop when you have a surgical robot is that not everything in surgery is predictable and something that we can control. Robots are really good for doing things repeatably and in very controlled, easy environments where things don’t change, but as soon as you start to have uncertainties – and in surgery the kind of uncertainties can be bleeding, it can be unexpected changes in tissue that, you know, that patient might have something that you weren’t expecting – and if I don’t program those kind of conditions for the robot to be able to avoid potentially adverse events, the robot’s kind of useless at that point. So that’s sort of why you want the surgeon in the loop. The surgeon has intuition, they have a more global understanding of what’s happening in the surgery, and so when something unexpected happens you want the human there to be able to prevent a bad outcome.
[Ricardo] So what role do you play in your research with surgical robots?
[Dr. Fey] So I sort of have two roles. They’re related, but they’re two distinct roles. So one role is designing the robot in the first place, and oftentimes surgical roboticists will meet with some clinician – it could be a surgeon, it could be an interventional radiologist, it could be a cardiologist – and they tell us, “Hey, I have this medical procedure that’s really difficult and I need some kind of special robotic tool that’s going to help me perform this procedure better.” So that’s one thing that we do is designing new novel robots for very specific surgical tasks, medical tasks. And then the other thing that we do is taking existing surgical robots and trying to understand how to improve them, how to make them more intuitive to use for the surgeon operator, and how to essentially program them to be safer for the patient in the OR by understanding more things about how the the surgeon is trying to use them.
[Danyelle] How do you overcome the challenges of the limitations of the robot?
[Dr. Fey] So the challenge with surgical robots – there’s a, there’s a lot of uncertainty happening. So the surgical robot is really good at doing things that are consistent and in sort of in a very simple environment where things aren’t really changing, so that’s sort of what the robots really good at. But the robots also really good at taking information and understanding things from that information through the use of some novel computer algorithms like machine learning or artificial intelligence. The human operator on the other hand they’re really good at understanding the big picture being able to prevent potentially bad things that might occur for the patient when there’s unexpected situations that occur. But the human also has lots of uncertainty so if we want to let the human help control the robot, now imagine that your surgeon is tired, that they drink too much coffee so their hands are jittering, you wouldn’t necessarily want the robot to do what the surgeons telling them to do in all cases. So, so one of the kind of exciting things about surgical robots is that we can actually design how the robot and the human communicate with each other. And that’s kind of what we study. How we can take information that we learn about the human and what they’re trying to do and then let the robot then decide whether or not they actually want the surgeon to do what they’re trying to do. So if it’s not good for the patient, if we see that the surgeon, that they’re having you know jittery movements or they’re kind of being undecisive or not very direct in how they’re moving we can actually tell the robot, “Hey don’t listen to the person right now and tell them to go take a break,” something like that.
[Danyelle] So then do you work with one robot and one surgeon to determine how that surgeon’s baseline is, or are the robots capable of recognizing uncertainty across the board with multiple humans?
[Dr. Fey] So that’s kind of what we’re trying to get at, is this understanding, designing some kind of algorithm for the robotic system that’s gonna work for all surgeons, you know, but where we start, we start with trying to come up with some sort of model of human behavior. So one example is trying to predict the difficulty of the task. We use this using measurements of how the surgeon is performing a task. We look at things like the velocity of their movements, how quickly they accelerate. We even measure things like their muscle activity and something called skin galvanic response, which is a measure of your skin sweat and anxiety. So we can take all of these measurements of the human and use them to sort of predict how difficult a task is and we’ve designed our models to be general enough so that it sort of trained on one set of people who have a big range of expertise but then I can use then that same model for a new person who may be, you know, I didn’t, I haven’t observed before.
[Ricardo] Yeah, that, I never took into consideration like, how like, you have to read the actual surgeon to like, you know, to like know how the robot’s gonna perform and all that stuff.
[Dr. Fey] Yeah so we, we do a lot of human experiments in the lab where we’ll recruit either expert surgeons or novices, even you know students at UT Dallas who have never done anything medical before, so that we can study essentially, their behavior in, you know, with our robotic testbed that we have in the lab.
[Danyelle] And how well do the robots do at quickly picking up what factors might be at play with the surgeons?
[Dr. Fey] So right now the robot doesn’t do anything like that because the, the models that we’re developing, they haven’t yet been programmed into the robotic system yet. But that’s, that’s where we’re hoping to go. You know, in terms of how well do these work, I would say at least in, in the cases that we’ve tried, pretty well. So, so as an example, some of our models, we can predict how difficult the task is using sort of this standardized difficulty index metric with, you know, close to 90% accuracy just to give, you know, a number to something. And then the other, the other neat thing that we can do is we can actually predict the style of expertise. So what I mean by that is, when you watch an expert who’s doing something – whether they’re a surgeon or a violinist or a basketball player – there’s something in how they coordinate their movements – they’re smooth, they’re fluid, they’re decisive and direct in how they’re moving – and we can actually quantify those stylistic patterns of expertise onto the robot and we can use that to predict when the surgeon either becomes more like a surgical expert or less like a surgical expert. And so in those cases, using our sort of stylistic prediction of expertise, we actually have a better predictive value in knowing how good our surgeon is than if you were to just take metrics coming off of the robot themselves so things like times and errors and and things like that we sort of have a more richer understanding of what it means to be a surgical expert.
[Danyelle] Dr. Fey works with students of varying levels – even those outside of engineering. She also collaborates with researchers at UT Southwestern.
[Dr. Fey] So my research team now – it includes graduate students. It also includes undergraduate students and even high school students. I take a lot of high school students in the summer through the UT Dallas NanoExplorers program. We’ve had awesome students in the lab that have written papers that have gotten nominated for best paper awards and they’ve really done some really impressive things. So that’s sort of what my, what the engineering side of my research team looks like – students at sort of all levels at UT Dallas. But we also have a lot of people on our research team at UT Southwestern. So these are our surgeon collaborators who help us define new projects and new directions for the projects. I work closely with the simulation center at UT Southwestern Medical Center where we do pretty much all of our experiments, is down there at the sim center [Simulation Center] and we also have even medical students that are on our team that do research projects with us, surgical residents and fellows. So it’s really a broad team of people that bring their own unique perspectives to surgical robotics, either through the medical side or through the engineering side.
[Ricardo] Tell us a little bit about your lab.
[Dr. Fey] I direct the Human-Enabled Robotics Laboratory at UT Dallas and we call ourselves the HeRoes. We actually just moved our lab to the new Engineering and Computer Science West building, so we’re very excited about moving there. We have an entire high bay that’s dedicated to medical robotics, which we’re very excited about. And in terms of the tools that we use, I’m also very, you know, happy to say that we have the only surgical robot in the world right now that you can actually program and control yourself. It’s, it comes from the company that actually produces it – Intuitive Surgical. They have a research version of the surgical robot, the same robot that’s used in ORs across the country and we have the only one in Texas here at UT Dallas and one of only 32 in the entire world. So I’m, you know, very proud to say that we have this great resource in the lab that we can actually program this robot and we can change how it responds to the human operator, which is really a huge resource. And in addition to that we have lots of other sensors that we put on people. We have EEGs, which is sort of the, it’s, it’s a brain monitor so, so we have that in the lab that we sometimes use. We measure muscle activity. We measure how they move with something called inertial measurement units. So that’s how we measure how the user moves and all of those sensor, all that sensor information just helps us get a better understanding and it paints a better picture of what the surgeon is trying to do and how well they’re doing it.
[Ricardo] At this point we wanted to get a visual of what the technology actually looked like. What exactly is a surgical robot, and how does the surgeon interact with it?
[Dr. Fey] Historically, when a surgeon did a surgery, and they still do this today, but from the beginning of time for surgery what that entailed was the surgeon physically put their hands inside the patient and they might use tools like scissors or a needle to cut things, sew things back together and then and that’s what basically surgery was. Starting in about the 1980s, different surgical tool manufacturers started coming up with these instruments that would allow the surgeon to do the surgery minimally invasively. So what that means is that now instead of the surgeon physically putting their hands inside the patient, they can do surgery through long thin instruments that they put through the abdomen of the patient and so instead of having these really big incisions that the patient has to now have and they have to have a big scar after this what’s traditionally called open surgery, now they can have this minimally invasive surgery where the patient has very small incisions in their abdomen. And that’s now sort of, you know, how surgery is done. So the problem with this minimally invasive, it’s called laparoscopic surgery, is it’s actually really hard to learn how to use these tools. It’s not very intuitive. Maybe in kind of the 2000s a company called Intuitive Surgical, funny enough, came up with this robot that allows the surgeon to do laparoscopic surgery where they’re sitting at a master console. They get to put their hands kind of on these grippers that represent how the tools in the, in the patient move. So when the surgeon grabs these manipulators and they move their hand to the right then the instrument in the patient is also going to move to the right. And so they essentially feel like their hands are in the patient but in reality they’re controlling this really fancy joystick, which is the master manipulator for the robot and that communicates through computer code to the robot that is actually the instruments that are inside the patient.
[Danyelle] And so what are the benefits of robotic surgery, minimally invasive surgery, versus open surgeries?
[Dr. Fey] So the benefits of the surgery now are that not only does the patient have these very small incisions, overall the healing time for the surgery is much less than it used to be. There’s less complication, there’s a lower risk of complications after the surgery. So for example infection of the surgical site, things like that. So less pain, less complications, lower, a shorter stay in the hospital for the patient. So those are all patient benefits but the surgical robot also has a lot of benefits for the surgeon, in terms of allowing him to perform surgery like he used to be able to do with his hands in the patient but instead doing it in a much safer and more minimally invasive way. And he can also sit sort of comfortably at a chair in front of this master console to do the surgery and so it’s not as physically demanding for him as laparoscopic surgery historically has been.
[Ricardo] Surgeries usually last, a lot, well, some of them last very long periods of time. Is there a thing where you can, like, a doctor can, like, kind of, like, tag himself in and like, take over the surgery instead of…
[Dr. Fey] Yeah, so, and that’s one of the benefits of having the surgical robot now is that, yes, this, the surgeon can take a break and maybe there’s another surgeon that’s going to come in to do a different part of the procedure. And also for training purposes the surgeon might stop and allow their surgical resident to do some easier part of the procedure so that the the resident can actually learn how to do the procedure as well.
[Ricardo] Cool.
[Danyelle] We went on to ask Dr. Fey if the rate of medical errors has decreased since this technology has been widely introduced. Here’s what she had to say.
[Dr. Fey] So you bring up a really good point that medical errors – human-generated preventable medical errors – is actually a really important problem in the way that, that medicine is done. It actually accounts for the third leading cause of death in the U.S. right now, besides cancer and heart disease, which is sort of surprising.
[Danyelle] Hey, just so y’all know, the statistic doctor Dr. Fey mentioned comes from a May 2016 study from John Hopkins.
[Dr. Fey] And there are some lawsuits happening right now in the U.S. related to the surgical robot and this is a huge problem that, you know, we as a society need to address, understand because there’s so many factors at play when it comes to patient health. And I think especially as the lines start to get blurred between, you know, right now the surgeon predominantly is responsible for everything that happens in the surgery – the robot will only do what the surgeon tells it to do – but there are also mistakes, there’s glitches that could happen with the robot. There’s bugs that could prevent the robot from doing what the surgeon is trying to do.
[Ricardo] So here’s a tidbit from a March 2019 CNN article: The U.S. Food and Drug Administration issued a warning in March to patients and doctors considering the use of robotic surgery for mastectomy and other cancer related surgeries saying that the robotic surgeries are not approved for the treatment or prevention of cancer. The FDA has evaluated and approved robotically-assisted surgical devices for use in certain types of procedures, but not for mastectomy, saying that their safety and effectiveness have not been established.
[Ricardo] And now back to Dr. Fey.
[Dr. Fey] And now kind of taking it to the research that we’re doing, where we’re actually allowing the robot to change how it’s controlled and we’re actually changing the way that the human and the robot communicate together, there’s, there’s potentially huge ethical considerations that we haven’t yet even begun to think about. So it is going to be something that’s really important as we move forward. You know, when sort of the lines of who’s responsible really become blurred the more that the robot has the ability to kind of control the situation.
[Ricardo] So. Time for the big question. Could we ever have robots that to do surgery without a surgeon?
[Dr. Fey] You know, I can imagine that you know, maybe 50 years down the line we could have robotic systems that are smart enough and that they, they are able to have access to enough information that they could potentially perform certain parts of procedures autonomously. But I think you really wouldn’t want to have the surgeon completely out of the loop. And the reason why is because, because that’s a human. They, they understand things in ways that our robot is never gonna be able to understand and at least for me, personally, I think I would always want to know that there’s a real person that’s there to make sure that my surgery is as, you know, safe and effective as it can be.
[Danyelle] So what do you think is the future of robotics in the medical field? Cause our main topic for today was, could we ever have surgery with robots alone. You said maybe, but you don’t feel comfortable with that.
[Dr. Fey] Yeah, and so I think the future of medical robots in healthcare – I think there’s two maybe important things. One is finding ways to provide better information to the robot about the patient, what’s happening to the patient. So integrating different types of imaging or sensing or anything that can provide the robot more information about what’s happening to the patient in real-time. And then the second piece that I think is extremely important is providing information to allow the robot to understand what the human operator is trying to do, and that’s obviously what I study the most. So I think is actually the most important part, and the reason why I think it’s actually the most important part, is that the human can also look at all the medical images, the human can read the heart rate data about the patient. So, so they kind of can internalize a lot of knowledge about what’s happening to the patient but if the robot doesn’t have a good understanding of what the surgeon’s trying to do and how well they’re trying to do it, then, then that’s always going to be limited. So, so that’s sort of why I think the most important thing to study is the human operator. At least at this time.
[Ricardo] Ann, what a delight.
[Danyelle] What a delight!
[Ricardo] thank you for being here…
[Danyelle] Yes!
[Ricardo] … and talking to us about things that we have no idea about. It was a pleasure.
[Danyelle] Yeah.
[Dr. Fey] Thank you guys so much for the invitation and to help spread the word about surgical robotics and engineering at UT Dallas.
[Ricardo] You can find more information about Ann’s lab at utdallas.edu/hero.
[Danyelle] If you want to check out her school you can find the Erik Jonsson School of Engineering and Computer Science on Twitter @utdjonsson. And Jonsson is spelled J-O-N-S-S-O-N.
[Ricardo] The UT Dallas CometCast is a podcast network brought to you by the UTD Office of Communications.
[Danyelle] A special thanks to senior lecturer Roxanne Minnish for our music. Be sure to follow the university on social media and check out Could We Ever and our other shows at utdallas.edu/cometcast. So, listen out for us next time.