Dr. Andrea Spiker...: Welcome, everyone, to the Arthroscopy Association's Arthroscopy Journal Podcast. I'm Dr. Andrea Spiker from the University of Wisconsin. Today, I have the privilege of speaking with Dr. Shane Nho, who is an associate professor and section head of young adult hip surgery, as well as the assistant program director for sports medicine at Rush University Medical Center in Chicago, Illinois.
Dr. Nho was a senior author of the publication titled Development and Internal Validation of Supervised Machine Learning Algorithms for Predicting Clinically Significant Functional Improvement in a Mixed Population of Primary Hip Arthroscopy, which was published in the May 2021 edition of Arthroscopy Journal. His coauthors on this publication include Kyle Kunze, Evan Polce, Benedict Nwachukwu, and Jorge Chahla. Welcome, Dr. Nho, and thank you very much for joining me.
Dr. Shane Nho: Thanks for having me, Andrea. A pleasure to be here.
Dr. Andrea Spiker...: So, Shane, to start us off, can you tell us a little bit about your practice and the evolution of this study idea in particular?
Dr. Shane Nho: Sure. So, I've been in clinical practice since 2009. I developed a young adult hip practice at Rush University Medical Center. And, in the outset of my practice, I had a goal that I really wanted to develop a robust clinical repository. And I think it takes a lot of forethought and planning, as you're about to get into practice, to think about what you want to collect in terms of outcomes, how you're going to do it, what software platform you're going to use, what x-rays you're going to get, MRIs, because you want to try to standardize things as much as possible.
So, I think all those things I had to kind of think about as I was starting, and what better time to think about that when you're just starting out in practice and your clinical volume is not very high. So, all the groundwork, I think, had to be laid out in the beginning.
And obviously, I had a lot of support from mentors like Bryan Kelly and Tom Byrd and Marc Philippon, who has shared with us and taught us a lot about hip arthroscopy in the beginning. And so, there was, obviously, a collection of papers that have already been published, and hip-specific patient-reported outcomes that were in existence.
And while it was a young field, there was, I think, enough, I guess, of a research foundation that really allowed me to kind of think about all those things and try to plan for the future. But, it certainly has evolved over time. Not just in terms of the research side of things, but obviously my clinical practice grew pretty dramatically from a very kind of even practice in terms of shoulder, knee and hip, and then evolving to practice, I would say, a majority of hip at this point.
So, I've been collecting data essentially since... My first full year was in 2010. And I guess the way I think about it is that the first couple years of my practice was a pretty steep learning curve, in terms of how to perform surgery in young hip patients. The technique was evolving, there was still a lot of questions. Things like preparing the labrum, how to accurately assess and comprehensively correct femoroacetabular impingement, capsular management evolved quite a bit during that time.
And so, there was a pretty steep learning curve, just in terms of our understanding of the disease, but also the surgical innovations, techniques, instrumentation that that kind of evolved, I think, during that period, too.
When I think about it, and when I talk to my research team, I always consider January of 2012 to be the onset of when we started to perform what I would call the contemporary treatment of femoroacetabular impingement. And so, in my mind, I always think, when I'm going through the database, this is when we want to start really focusing our collections. Because it was the confluence of the maturity of my own practice, the establishment of our research, protocols, and also I think the evolution of the surgical technique in terms of what we were doing. And that was obviously predicated by our improved understanding of the young adult hip.
Dr. Andrea Spiker...: So, before you began this particular investigation, what was your understanding of machine learning and its use in medicine or orthopedic surgery? And what do you think it is about hip arthroscopy that makes the world of hip preservation the next frontier for this machine learning exploration?
Dr. Shane Nho: I think my introduction to machine learning is probably similar to everybody else. With a lot of this big-data way of kind of processing information these days, whether it's via the internet, or Google search engines, or Facebook, or whatever it is, I mean, I think artificial intelligence and processing of lots of data, I think, is basically the world that we live in at this point.
And I think I learned about it just as we all have, just because we're part of it. And so, I think one of the things that set us up for this is that we had a pretty complete data repository. In doing a lot of prior studies, looking at our database, we'd done pretty sophisticated statistical analysis, modeling, regressions, and so forth.
And I think it really took the lead author on this, Kyle Kunze, who had a special interest in machine learning and machine learning algorithms, to really apply that to our dataset. And I do think that machine learning is pretty exciting for the world of orthopedic surgery, hip preservation, but also, I think clinical medicine in general, because I think the cool thing about is that, in my mind, it basically... I think as high-volume practitioners, like you and me, I mean, basically we're making a lot of these predictions based on our pattern recognition.
So, basically what this is doing is it's using data from our repository and calculating the predictions of being able to identify what patients we think are going to do well. You and I see patients all the time, and we're basically trying to look through their clinical information, their physical exam, their x-rays, or MRI, and really just process everything in that exam room with them and decide whether or not we think that they're better treated with either nonsurgical methods or surgery.
And then if we do think that they might be a candidate for surgery, I think in our own minds, we're kind of calculating, ourselves, how well do we think this patient is going to do? And how do I talk to this patient about the surgery? Do I give them the hard sell, like, "You're going to do great," or do I lay the crepe and say, "Well, you may or may not do great"?
And I think that we're all basically doing this. We're doing it with our own experience, with our own kind of individual datasets, and we're doing it by pattern recognition. But I think it's much more powerful when we have the data behind it, to be able to say with, I would say, more confidence, not just for ourselves, but for our patients.
To say, "Look, we've done these studies, and we do these machine learning algorithms, then we can actually predict your potential outcome after this. And I think that that's the exciting thing about the next, I think, evolution of medical decision-making and treatment decision-making for patients.
Dr. Andrea Spiker...: Yeah. I absolutely agree with you. And I think that, circling back to something you said earlier, where it takes you years to get to that place in your practice where you can even begin making these decisions or recognizing these patterns.
I think the potential for the next generation of hip arthroscopists is really fantastic with this type of application, where perhaps we can cut down on that learning curve for all of the learners and trainees that we're trying to bring into this hip preservation realm as well.
So, you've mentioned this repository data collection, and you have published many fantastic studies looking at patient-reported outcomes in hip arthroscopy patients. And, in fact, you're a repeat guest on the Arthroscopy Podcast, and last time you were here, you discussed one of your studies looking at the preoperative predictors of achieving clinically-significant athletic functional status after hip arthroscopy.
So, you've mentioned it a little bit before, but can you really highlight your thoughts on the importance of collecting patient-reported outcomes, and then mention some of the challenges that you've encountered in collecting such a vast amount of data on your patients.
Dr. Shane Nho: Collecting patient-reported outcomes, I think is really extremely important for individuals, I think for, let's say, our section, our department. For orthopedic at large, I do think that it's important to collect something.
It doesn't have to be five different hip-specific or disease-specific patient-reported outcomes, but I think you're only able to learn if you are able to kind of reflect back on your own patients and see how they do. Because otherwise, you can pick and choose your good cases and bad cases. And I think it just helps you to evolve as a practitioner if you can really understand your data and understand how your patients are doing.
So, I encourage everybody to do it. I think that the challenge of it is that it's not easy. And to be quite honest, I have a research team at this point. I mean, we've got three full-time researchers, one in medical school, two just after college. We've got residents, we've got fellows, we have medical students. We have research coordinators. We've got a lot of people that are participating in this.
We also have evolved in terms of how we did it. I mean, when we were first starting out, it was literally an Excel spreadsheet and paper surveys. And then, we moved on to a different software platform. Now we're on a second software platform, and I'm hoping that each evolution of this, that we obviously are able to maintain that core data that we've had since the beginning, and then apply it to more sophisticated ways of collecting the data.
The data is difficult to not just collect, and I think that's the biggest hurdle, but also to tabulate it or to keep it in a repository that we can query pretty easily and not have to process the data for every question that we're doing. I think our latest repository does give us a little bit more sophistication with that. It not only holds our data, but then allows us to analyze it real time. So, I think that's pretty cool.
So, having software and technology and basically investing in it, I think is important. I mean, like I said, it's hard to do it with paper charts and Excel spreadsheets. It's just much more difficult. But I think that the hardest thing, and I think that we struggle with the most, I think most people do, is getting the follow-up.
And I keep telling my research team that the part that's missing with clinical research is engagement of patients. Because we basically collect all their data, and we just don't... We collect them, and it's so siloed. We don't really engage them. The cool thing about some of these machine learning applications is that we can actually share with them, look, your data has as allowed us to be able to create these algorithms. We can actually calculate real time how you're doing. We can benchmark you against your peers.
And that's the missing part of it that I think would help a lot, is that one of the software programs, for example, the patients fill out their PROs pre-op, three months, six months, one year, two years, five years, and so forth. But we see them at the three-month and six-month visits, sometimes the one-year visit. And if they filled out patient-reported outcomes at all those time points, there's actually a real-time chart that will be displayed on the program, so that I can say, "Mr. So-and-so, here's how you're doing since before surgery. Look at how much you've improved." And then we can also compare them real-time to their peers, based on age, or gender, or whatever.
And so, I think that that's the part that's really missing, is that how do we engage patients and get them excited about it, rather than burdening them with all these questions. But I think the other thing is really trying to be sensitive to their time and their commitment and what they've given to us, and coming up with patient-reported outcomes or scoring systems I think are more efficient. For example, using Promise and other health metrics, rather than saying, "Look, you've got to spend 30 minutes to fill out all these questionnaires."
Dr. Andrea Spiker...: Yeah, I agree. And I have many of the same troubles in collecting patient-reported outcomes, which I also believe are so important. And I think we've talked a little bit about the importance of learning about this surgery, and how to do the best we can for the patients, so it's helping us as practitioners, as surgeons. It's also helping our trainees. But I think you bring to light the most important fact, that it's really helping our patients, ultimately, because we're getting better at this and at addressing their pathology based on this information.
And I absolutely agree with engaging them more. I think you're spot on. That's how we're going to get them to follow up and follow through with these outcomes.
So, as part of this investigation, an open-access digital application was created. Can you tell the listeners a little bit more about this application? Was it intended more for surgeons who want to evaluate patients, or is it more for the patients themselves to use?
Dr. Shane Nho: I mean, it could be for both. It's pretty cool. I mean, so the web address, I'll just kind of shout it out to the listeners, but it's https://orthoapps.shinyapps.io/HPRG_ADL. It's on the article. The amazing thing about is that it's a website, and you can go on it now, if you wanted to. And what you basically do is, you can input the features or predictors that we have found to be significant from this paper. And those include HOS-ADL preoperative score, gender, whether or not they had preoperative duration exceeding two years, BMI, their preoperative VAS pain score, whether or not they have a drug allergy, their age, and their Tönnis grade.
So, let's just hypothetically come up with some numbers here. So, let's say their pre-op HOS-ADL score was 65. They're male. They did not have more than two years of pain. Let's just say they've got a 24 BMI. Their VAS pain score, we can say a seven. No drug allergies. We'll say they're 25, and they have a Tönnis grade zero. Okay? You got all that, Andrea?
Dr. Andrea Spiker...: Yes.
Dr. Shane Nho: So here, and again, imagine sitting in the office with a 24-year-old guy, normal BMI, high pain score, they haven't had symptoms for that long, maybe six to 12 months or something, and they don't have lots of drug allergies. But now I just click the button now, okay, and in real time, the model prediction helps us to determine what is their probability of achieving a clinically significant improvement. And they're saying that this person, with these characteristics, has about a 96.2% chance of achieving MCID on the HOS-ADL.
Dr. Andrea Spiker...: Shane, I'm following along with you on the website right now. It is incredibly easy to use, and that is pretty awesome.
Dr. Shane Nho: So, do you see the explanation down below that will show you how they weighed all these different characteristics?
Dr. Andrea Spiker...: Yep. So, the red and blue bars. That's pretty cool.
Dr. Shane Nho: You can use this to have a conversation with the patient. Because this patient's kind of a no-brainer in my mind, and I picked this intentionally, because you know that a young male, no arthritis, pretty active, they haven't had symptoms very long. Again, what we're basically doing in our own minds, this is just kind of putting it down on paper and saying that, "Hey, you've got a 96% chance of achieving MCID. This surgery would be very helpful for you."
But, take a different patient. Let's say you go to door number two, and let's say you put in different data. Let's say they've got an HOS-ADL score of 40. They're, let's say, female. They're overweight, let's say a BMI of 35. Pain score of about a nine. They've got drug allergies. Let's say they're a 50 year old woman with Tönnis grade one, and they've had pain for a long time.
And these are all the things that we think about, in terms of this is someone that we really don't want to operate on. And so, it's giving me a 59% chance of achieving MCID. And when you look at the bars, I mean, it's basically all red. It's like, "Don't operate on this person."
Dr. Andrea Spiker...: Yeah. What a valuable tool and an excellent use of technology. And I think the patient education output as well. So, that's really quite awesome.
Dr. Shane Nho: Yeah. And so, I think it's that, and also having that shared decision-making with the patient, and giving them the data up front, and just saying, "Look, here's how we think you can do based on the characteristics that you've described. You can do surgery if you want. You don't have to. This is how we think."
And honestly, we're actually trying to put together a randomized clinical trial, where I would either talk to a patient, and either use this program as a way of counseling patients and seeing if that affects your decision-making, versus just randomizing into that, versus me just saying, "Hey, I think you'll be 90% successful in terms of reaching clinically significant improvement." And basically seeing how the program... My thought is that, if you were a patient and you saw this, this would make you more convinced to do something or not, versus you or I telling the patient.
Dr. Andrea Spiker...: Absolutely. Yeah. It's based on data, which you have worked so hard to collect. So, I think this is a really very awesome application and utilization of our current technology for all of those reasons, and patient education. So, congratulations to you guys. This is really quite inspirational.
So, speaking of, so it looks like these factors that you were listing in this application are based on the results of what you found to be the most important features for predicting MCID. So as you mentioned, body mass index, age, preoperative hip outcome score, pain level, gender, and then Tönnis grade, symptom duration, and drug allergies.
So, how did you decide, when you were inputting the many different variables that you had on patients, what to include to come up with these eight most important factors? I mean, was it a large data dump? Did you just take every piece of information about the patient? Or where you culling the data to input just a certain amount of information on the patients to see what was most important? Did you have some sense of what your results would end up being?
Dr. Shane Nho: Well, I mean, initially our featured selection consisted of 21 different features or factors. Some of them were known. We had a variety of both modifiable and non-modifiable factors. And basically, the features or factors that were eliminated were basically run through this random forced feature elimination program. And so, this program basically fits a model, computes an importance score for each feature, and the ones that are least important end up getting dropped out. So, based on all that, then they came up with the final features that we had described.
Dr. Andrea Spiker...: And, what do you think about those results? So, were those eight features what you also found in your practice to be the most important? Or were you surprised by some of them that ended up being more important than others?
Dr. Shane Nho: Yeah, I mean, I think that these are ones that I would... And we published other clinical studies looking at other predictors not using ML, but just using either some sort of regression analysis, or even comparing matched cohort.
But, I think age comes up as a pretty frequent predictor for hip arthroscopy. Gender is one that I think is always described. BMI. So, I think a lot of these... Tönnis grade, preoperative PRO score, preoperative pain, duration of symptoms. The only one that I think was a little bit surprising was the presence of one or more drug allergies, but I think all the other ones are ones that... I think you would probably agree that these are ones that you think about when you're seeing patients in the office, and ones that you know that will either be a positive factor or a negative factor.
Dr. Andrea Spiker...: I agree. I think based on our clinical experience, that's what's come up, and it's really interesting. And also encouraging that the machine learning algorithm found the same.
To finish up our discussion here, I mean, we've talked about this really incredible innovation that you and your team have come up with using years and years of data, and now also incorporating current technology. But where do you think our field is headed next? I mean, is machine learning going to become more a part of our practice, or is this simply an aid to help us accomplish other areas where we're going to excel in hip arthroscopy? What next?
Dr. Shane Nho: Well, yeah. I mean, I think predictive analytics would be the goal. I mean, if we can predict patient outcomes before they have surgery, I think it'll be a tremendous value for providers, obviously patients. And fortunately, I think there are other stakeholders that would also be interested, hospitals, insurance carriers, and so forth. But, I think that this is basically continuing that shift towards value-based care, where we could predict outcomes, and the other people that are looking at costs, and how do we predict this so that we can basically provide value-based care for a population of patients? Rather than, I guess, fee-for-service, or in some cases over-utilization. Maybe we shouldn't be authorizing surgery for patients that we know aren't going to do well. It could be a patient benefit. It could be a society benefit. There could be some healthcare savings involved.
So, I think there's a lot of stakeholders here, and I just want to be sure that as physicians, that we are the ones who obviously are making those decisions, and providing that information and data, so that we can use it to best provide cutting-edge care for our patients. And I think that's the way I would see it, because if we don't do it, someone else is going to do it for us.
Dr. Andrea Spiker...: Right.
Dr. Shane Nho: And so, I kind of feel like we should get a handle of our own data and be able to make those decisions in the office with our patients, rather than having other people make those decisions for us.
Dr. Andrea Spiker...: Yeah. Absolutely. And I think there's always going to be a role for our clinical judgment, no matter what these machine learning algorithms tell us. And so, I agree. It's great that you're taking the steering wheel here, and heading us in the right direction.
Well, thank you so much, Shane, for sharing your thoughts with us today. It's been a true pleasure speaking with you. Thanks again for joining us for the second time on the Arthroscopy Podcast.
Dr. Shane Nho: Thank you, Andrea. A real pleasure to be with you and the podcast, and hopefully we'll get invited for another one.
Dr. Andrea Spiker...: Dr. Nho's paper titled Development and Internal Validation of Supervised Machine Learning Algorithms for Predicting Clinically Significant Functional Improvement in a Mixed Population of Primary Hip Arthroscopy can be found in the May 2021 issue of Arthroscopy Journal, or online at www.arthroscopyjournal.org.
This concludes our episode of the Arthroscopy Journal Podcast. The views expressed in this podcast do not necessarily represent the views of The Arthroscopy Association or the Arthroscopy Journal. Thank you for joining us, and see you next time.