When Graphs Don't Work

The content discussed in this episode may be specific to the guests organization or location and may not be applicable to other organizations, states, territories or countries. The material presented in this podcast is for general information purposes only and should not be considered health, legal, or financial advice. Administration is a rewarding profession, but it comes with challenges. So what is it like and what should you do if confronted with a monumental administrative difficulty?

This is the podcast. Takes unfortunate scenarios. and explores the essentials of clinical governance to find solutions. I'm Dr. David Rankin, a very experienced medical administrator and lead fellow with the Royal Australasian College of Medical Administrators. And this is Safeguarding. The essentials of Today, how not to confuse your surgeons. At the end of the day, we employ them for their surgical skills, not for their data analytical skills.

That's Dr. Craig Marguetz, a veteran medical administrator who has held senior positions in a range of hospitals across four Australian states and territories. And I'm presenting him with a difficult scenario. gone down a path without taking enough care and diligence to make sure that it's meaningful for them and in a way that they can do something about it.

In today's scenario, as the Director of Medical Services in a large public hospital, you've been creating and sharing individual comparative reports for your orthopedic surgeons for several years. You are proud of the way you have been able to identify clinical indicators, creating variance reports from your hospital's administrative data. You believe that data presentations are intuitive and well laid out.

At last night's orthopaedic department meeting, one of the busiest surgeons mumbles in a stage whisper, that they really like receiving the reports but they have no idea what they mean. This creates general chuckles around the room, with other specialists seeming to agree with the sentiment.

I start by asking Dr. Craig Marguetz what issues are raised in this scenario. I think it's really important to understand that the Gulf between the way clinicians think and the stuff that's important to them. And sometimes the way that finance and administrative people look at information, there's A lack of understanding of what's maybe important and an assumption about knowledge, for example, or frequency of which people have been looking at reports.

This would concern me because it means that people that the clinicians that I've been sending these reports to, I've thought that they were looking at them and taking action, but maybe they haven't been understanding it. Maybe I've

gone down a path without taking enough care and diligence to make sure that it's meaningful for them and in a way that they can do something about it. So we often assume That doctors are intelligent, they're trained, they understand data, but many we find don't have an intuitive understanding of graphs or tables.

How can we be assured that the doctors understand the material that we send to them? I think that it's fair to comment that the doctors that we're dealing with are intelligent people and and in fact that's one of the challenges because Of their expertise in one area. Sometimes they're reluctant to to reach out for help for understanding. But it also helps greatly if

The information that we're providing is something that's recognizable. So for example, a practical case when we're talking about uh ICD data, international classifications of disease data, clinicians may not be familiar with that. So a technique that I've used successfully in the past is find a recent case, something that's that they patient they can recall, and then take them through what the data shows for that.

patient. And often you'll find the light bulb moments when they'll put two and two together and the clinical information and the administrative information start to come together in their minds. That then gives them the ability to ask more detailed questions about what it is they don't understand and that then can uncover shortcomings in my assumptions of what I thought they might be interested in and what I thought they might understand and look at

So it's always an exercise where you need to make sure that you're not having a senior and a respected professional loose face to make sure that the information you're providing can be tracked through to something they might want to and something they are able to do something about. So for example,

when talking about comparative data about complications. One of the common things that I've found over the years is the difference in the definition between what a surgeon will think is a complication and what the International Classification of Diseases regards and calls a complication. And that can cause quite offense.

when a condition that might have occurred anyway, a patient might have developed uh atrial fibrillation completely unrelated to the surgical procedure and might have happened any time. And the fact that you can decide Explain that Complication from the international classifications of diseases and the WHO perspective is simply a diagnosis that wasn't there on admission. And sometimes you could just reframe it or rephrase it.

in a way say these were diagnoses on admission and these were the new diagnoses that happened, some of which we would be concerned about and some of which we wouldn't be concerned about. So different ways of finding ways of explaining the information and the data in ways that make sense. To the clinician.

I think that's a really important issue. Engaging the doctors. So often we send out data and information that we want them to have. We don't sit down first and ask what information they want to receive and what the implications for that data is for them. So what are you going to do with this group of orthopaedic surgeons that nervously laugh and agree that they really don't understand what data means? Well I'm gonna start by apologising because at the end of the day we employ them.

For their surgical skills. Not for their data analytical skills. And it's my job and our job to make that easy for them. So I would like to think the surgeons that I've been talking to that I wouldn't be in this situation because I would have started by asking them the question. But here we are in the situation and finding common ground, finding areas that they're interested in and seeing if we can align the information.

There's other ways of putting it. So, for example, many surgeons will have their own complications data that they'll keep themselves. And so you can say, well, So you've got some information about some things that you're interested to look at. Can we look at some of those and then see how they are being reported on your behalf to make sure

that we're doing the right thing by you. And so approaching it as saying, your data's right, my data might be wrong. I'm just trying to get your help to help with my data. I've also used a technique where when a report might say that there's a a problem or a variance. It's about being a little bit an approach of humility and saying, Look, I'm not sure this data's

correct, but I'm not necessarily going to pick up where, but I know that you'll be able to pick it up very quickly. Can you have a look at this with me and show me where the data's incorrect? So That approach first of all gives them permission to say the data's incorrect without fearing that they're going to offend you. And I've been kind of surprised that often they'll go

Yes, well the data's wrong here and here, but actually over there and there and there it's actually quite good, really. And you can hear the slight surprise in their voice and where the data's incorrect you can say, Well how can we is this a definition? Or is this is it really incorrect data or does this mean that there's a problem with the documentation, for example, the junior doctors are

not writing as well as we could. How can we work together to make sure that we're representing the complexity of the work you're doing and the excellent work that you're doing? So I guess one of the aims of sharing data with individual specialists is to get them to reflect. What are some of the barriers in getting a busy surgeon to reflect? What are some of the, I guess, emotional or

barriers that they might have. If you say to a senior doctor, or a junior one for that matter, that you believe there's a significant issue, this is non trivial conversation. You're cutting to the core of who they think they are as a human being.

what their life is. And so you need to approach it with enormous care and caution. Another one of the reasons why I I tend to approach it by starting with saying I think the data's wrong but I don't know where is to try and minimise any perceived threat in that'cause it can be terribly threatening. And I tend to rely on

the internal competitive nature of doctors. Uh if you provide information to doctors that they believe is credible and they've done the reconciliation with some patients and you help support them through it. the improvements will occur in the vast majority of cases where those improvements are required. Those improvements will just occur almost as if by magic, with very little further inter intervention, just by getting that relationship with the clinicians and supporting them with the data.

You should focus on individual reports or peer group reports. Do you provide the reports to the M and M meeting and they can all discuss it together? Or do you provide it to the individual doctors or both? So what's the comparative value of group reports versus individual reports? I think it depends a bit, David, on the data that you're talking about. If you're talking about length of stay, we know that length of stay data's pretty robust.

People are discharged when they're discharged. It's fairly reliable. The chances of it being widely different from reality is lower. And the interpretation is a lot simpler. So I'm usually pretty comfortable about saying, Well, here's some comparative length of stay data. We might can be comparing our hospital or our unit with the health round table, for example. And I just say, uh, is there any interest would you would you like to see how yours compare with others?

And generally they'll say yes'cause they all think that they're going to be better than average. Everybody is. And I'd start by sending out de identified information to each of them. I'd say, Well, you're this one, but the others are not identified. knowing full well that within about half an hour everybody will have figured out who everybody is, but I would also have gone through and made sure that there's not some

major embarrassing disaster there. In which case I would never do that in a public performer. That's always definitely a one on one. But for something like length of state, there's usually not a great variation and where there is a variation, there's usually a good explanation for it. And much of the time the variation is things like I couldn't get access to the rehab people or I had trouble with discharge planning and

Sometimes I found that the clinicians are sharing the frustration and they're pleased that you're there to help. But if it's about wound dehistance rates, which is a bit more sensitive from a personal point of view, or infection rates. I'd be very much dealing with them as a one-on-one until the difference between the best and the worst is only a little bit. Specialists often want to know who the patient is.

So you give them data, average length of stay, there's clearly one or two outlier patients, and they say, Give me the patient name. Is there sensitivity around I guess including patient details in the reports that you provide to specialists? I'm really glad you raised this because it's one of my particular passions that if you want to engage doctors, there's a fundamental thinking difference between

Clinical staff and administrative staff. Clinicians think in an N of one. It's one patient, one patient, one patient. And administrative staff think that if you do a nice big aggregate report, that'll mean more. Clinicians need to approach it from uh patient by patient and work up. Uh whereas the finance and the quality team will often work from an aggregate and work down. So to answer your question I would always give them individual patient data.

But I might de identify it. But you can re identify with a UR number and that sort of thing. Most of the reason I would de identify it is really because more to do with control of the reports and making sure that they don't, you know, fall off the back of a truck and that sort of thing from a patient privacy perspective. But if I said, Oh look, David, you know, there's uh a patient that had this particular procedure who was a you know, a seventy five year old lady, I go, Oh, that was Mary Smith.

So they can often identify the patient but between the diagnosis and the approximate time and the age and the gender. That's usually more than enough of them to know who we're talking about. But we did uh one point develop a a suite of reports that had, for example, all the I C D codes for the patients and there was a hyperlink that would link it through to the electronic medical record if they wanted to work it out. But it meant that it was de identified from other points of view.

But yes, your point is really important. You must start with the individual patients. And once clinicians have gone through ten or fifteen individual patients and found that they actually do match their experience. And assuming there's not some major problem, which sometimes there is, then they will trust the aggregate data.

So the other side of the confidentiality side is the chief executive. decides you're producing great reports for the orthopedic surgeons, they ask you for a copy, so they can identify the surgeons that are making a profit or losing the hospital money. Should Chief executives get access to individual doctors' reports.

Well it depends how much I like the chief executive because if I really didn't like the chief executive I'm giving the report he'd get into all sorts of problems and the doctors would come and lynch them and I'd make sure that the doctors didn't find out that I provided it. It actually doesn't matter. If the chief executive He's a non clinical person.

then, you know, it's hard to do that without sort of getting them delving in clinical matters to which they really shouldn't be looking at. I had a particular issue where where a board member, who also happened to be one of the doctors in the hospital, demanded to get patient reports about somebody else's patience and I had to stand firm. That was a somewhat challenging

conflict of interest on their part that I had to negotiate through. But the second part about it is the usually the chief executive can find out the data through the finance department or some other way. So

It's really about saying to the chief executive, let's focus on the outcomes that we want. Let's focus on uh getting everybody good. And then you don't need to worry about who's bad. So give me some time, negotiate for some time to identify and resolve the issue. Again in my experience, a good proportion, if not seventy percent of the time when there's a an issue like that.

it comes down to better documentation because actually the patients were very sick and the reason their length of stay is high is they've got very sick patients that the documentation isn't recognizing. So let's fix that first and then we'll bring everybody together. Because you're right, it can blow up in your face or or in the chief executive's face. I've always made sure that the reports I give to the chief executive are very different.

the reports I give to individual clinicians and making sure that I know exactly what the chief executive's going to do with the report and making sure that it's fit for purpose and the chief executive understands. the sensitive nature of some of these comparative reports. It's a tricky issue. Fortunately, my experience with chief executives has been largely on identifying things at the aggregate level and they've expected me to go and sort out the details.

So I've been fortunate like that. But there have been a couple of occasions where they've wanted more detail than i is handy. So as you say, providing them with a different type of report is one way through that. One of the dangers or potential dangers of producing reports like this is that they could be used in the media or the courts. Are the reports discoverable? Well, short answer is yes. In the sense that, certainly in the public sector, most of the information that you produce for

Even for an M and M committee or a quality committee that is protected, the fundamental data you get it from is not. So somebody who's clever enough can usually reproduce it, but it's difficult. You can protect in ways uh such as official quality reports, quality committees. Uh most M and M committees are not quality committees, but you can protect it that way. I must say I don't tend to go down that path because

Th it's better off if you're solving the problems rather than trying to, you know, hide them and cover them up, which is really what, you know, the fear is. Making sure that you're on the front foot is, I think, a far better strategy.

Thanks, Craig. It's an interesting issue. On the one hand, it's vitally important that each individual clinician understand what their performance is and how they compare, but each clinician's... trying to do their best and can be very threatened, particularly if their colleagues or their management team find out before they've had the opportunity to correct their issues themselves. Did you have any final thoughts on the issue of

Generating reports for clinicians? I'll finish with one little anecdote which was a true story. We had in a hospital I was working in a very high mortality rate. And when we looked in the mortality rate, the mortality rate was extremely high in the medical area. And the added complexity of that is that the head of the department was a very well known and very well respected across the country position of great importance.

So to suggest that that physician was in charge of the unit with the highest death rate uh across the country would have been yeah, challenging to say the least. In the initial the approach was to say that uh look, we've got this report, I don't think it makes sense, I don't understand why, but I'm sure you guys will know what we can do about this'cause clearly this is not who we are and

No, we we do good medicine. And it fairly quickly came out that because the palliative care unit didn't admit patients after hours or on the evenings, on the weekends, that all the patients who would have been going to pallid care were actually being admitted by the physician. So they were all going in there and then being transferred to Palcare and passing away.

And so we worked with them to say, Well let's assume that that's the problem, so see if we can take the data back, take out the patients who are known to PowellCare and then represent the data. And the clinicians came on a journey with us on that information.

in that process we also leveraged off that relationship we build with the medical records department and not only do we improve the quality of the data, we stopped the unit being potentially looked at by somebody in the media, as you say, as being some horrible unit, to being a really well performing unit.

And in the process we found an extra four and a half million dollars worth of DIG revenue. So there are success stories in that that you can take heart in, but again it's working with the clinicians rather than against them.

That was Dr. Craig Marguetz, a veteran medical administrator who has held senior positions in a range of hospitals across Australia. And that brings us to the end of this episode of Safeguarding Healthcare. I'm Dr. David Rankin. Follow Safeguarding Healthcare free on Apple.com. Google Podcasts, Spotify, or wherever you listen to podcasts. This podcast is produced by Sound Cartel for the Royal Australasian College of Medical.