Pitfalls: Errors in Radiology- Scope of the Problem

Introduction to the Meeting and Overview

I'm gonna take somewhat of a different take.
Don't panic, Mike, on this whole issue.
And I wanna sort of step back
and look at where error occurs all over the place,
if you will, in radiology.
And you'll see what I mean by that shortly.
The next couple of days, we are gonna be going
through most of these.
I think people, the excellent speakers who are here,
will go and drill down into these specific issues
and questions in their talks.
And of course, the whole goal, I think,
for this meeting is to see
how we can do a better job in preventing error.
And I would just like to sort of second what Mike said is
that I don't think I've been to a conference like this
before, which I think is, if you think about it,
it's a little bit staggering.
I mean, we can go to a thousand conferences
and how to read a CT or pet ct,
or I notice is a vascular ultrasound
something going on here?
And I'm not gonna knock it. I mean, we should go
to these things, but frankly,
how many courses have we been to on error?
Um, which is dedicated to error? I can't think of one.
Yeah, they're dotted in and out of these meetings.
And you can go to a,
probably a refresher course at rs NA on this
and wherever you are, but it's kind of,
you know, here and there.
So all credit to, to Mike who,
really came up with this idea.
So I'm gonna, I'm, I'm not gonna do much audience response.
I'm gonna ask one question and you can pretty much,
stick your hands up when you like,
'cause I can't see whose hands are going up from up here,
but I'm gonna ask you a question
and I'll just want some hands to go up.

Audience Poll on Addressing Error

So this is error in radiology now, okay?
How, how are we doing at addressing
error in our departments in our day-to-day work?
And I've got four answers.
Who, who's gonna stick up their hands for very well?
We do a, we do a great job in dealing with error. All right.
I don't, no, I want everyone to stick their hand up, right?
So I'm counting. It's, it's, you know,
we haven't got 3000 people in here,
so I can see everyone now that I've, I've got my cap on.
Alright, hands up for quite well. All right, good.
We've got about six or seven, maybe a few more.
What about Paul?
Okay, that beats the quite well, I think we have,
probably 60% in the audience.
And how about very poorly? Two, one. All right.
Okay. I'm gonna ask the same
question at the end and see what you think.
Alright, let's get into the, the, the beef of this,
what I'm trying to talk about today.

Peer Review and Broader Workflow in Radiology

Now I'm gonna call peer review a little bit more
as I'm going to show you on the next slide than just
interpretation, error, or Ms. Case, et cetera.
In a way, peer review I think should be,
as I'll show you about looking at our overall workflow
and product in radiology and everything we do.
So we really gotta kick up the backside, if you will,
in 2001 with the as, which we're all familiar with the IOM,
output, which said, we are probably killing the,
the operative word killing a hundred thousand people a year
through medical error.
In fact, they didn't really, really know
how many people were dying, because of this.
But that was kind of a guesstimate through mistakes, some
of them, crass mistakes, many of them through,
good intention, et cetera.
But there were still mistakes.
And the bottom line is, we are in a profession as much as we
believe, and I believe I can't, I haven't yet
to meet a physician who feels they're there to harm people.
As much as we believe we're here to help
and aid people, we do a lot of harm, a lot of harm.
And people come into hospital
with maybe a relatively small problem
and they go out having had a
myocardial infarction or something.
Now, it's interesting, if you step back,
whatever you think about Obamacare
and all the rest of it, I'm not gonna get into that.
The reality is the big driver
of Obamacare is about outcomes.
And outcomes is gonna fundamentally change the way we
practice medicine and fundamentally change the way
we practice radiology.
And you've begun to hear this term from volume to value.
So we've done extremely well, of course,
as radiologists in the volume paradigm by anyone, standards,
we've become very wealthy, even academic radiologists
compared to the average person in the population.
I don't wanna hear any complaints from anyone
that we're not getting a good salary
'cause we are compared to
what people out in the street are having to go and do.
And we've done very well under that volume paradigm.
But that's shifting now to the value paradigm where we need
to deliver value and better outcomes.
An error is a huge component to that.
And I would, I'm biased, of course,
but what I'm disappointed by in a way, is
that in some ways this meeting should have about 5,000
people in it,
rather than the 4,960 people who are going off
to see another talk on how to read ct.
Because this is one
of the huge drivers in the huge areas we need
to address over the next decade or so.

Deming's Principle and Knowing What to Do

So how do we do? Well, I do my best.
This guy, Deming, who was,
as many of you probably know, instrumental in looking at,
workflow productivity and best practices,
particularly in Japan, said,
it's not good enough to do your best.
Think about it. You must know what to do first
and then do your best.
And I'm gonna propose during this talk that it's very much
around that principle.
We first gotta know what we gotta do and then do our best.
We're human. We all underperform.
But I can tell you, we can do
so much better through knowledge.
We're gonna make mistakes.
And I've heard this from day one as, as a residency.
If, you know, if you're not making mistakes,
it's because you're not doing enough.
And the more we do, the more mistakes we're gonna make.
So I'm gonna propose to you
that error in radiology is not just about findings
or interpretation, error, it's actually much more than that.
It's about our overall work product.
And it affects every aspect of our work process.
And each component that goes wrong contributes, if you will,
to a suboptimal work product.
And all of that is error. It's waste in, in the process.
Many of you have looked at lean and,
exceptional.
The rest of it, the goal there is
to remove waste from the system to deliver a better product,
which is safer and high quality.
That's no different to what we're talking about here today.
ERA is waste in the system.

The Radiology Value Chain and Potential for Error

Now, what I'd like to propose to you here,
and I'm not gonna get into the weeds of this,
and it's partly from a different talk,
but one of the talks I'm giving at the moment is about the
radiology value chain.
And that's jog. And my, my wife,
who's a primary care physician tells me
to shut up every time I mention words like this.
But, I at least think it's helpful.
And a value chain is really something that,
is a chain of activities that in industry performs
to deliver a valuable product or service.
And if you think about what we do as radiologists,
we often think we're down here, right?
In reporting and communication
and distribution, distribution.
But frankly, we have to own that whole process.
I mean, I'm stating the obvious of course.
But we have to own imaging appropriateness.
We have to own the imaging protocols, the modalities,
the customer service, the throughput, everything.
And at each step in this, in link, if you will,
in this value chain, there's, there's,
potential for error.
It's not just down here in the reporting and the,
and the communication distribution.
It's at every step of the process.
'cause if you get it wrong up here, you order the wrong test
out of the gate, you've created the error.
I don't care how good you are at reading that ct.
If, if it's, come, come through to you,
if it's was never indicated in the first place,
that's a major error, right?
So that's what I'm gonna be focusing on.

Knowns and Unknowns in Error Recognition

Now, we all know this metaphor, right?
I think I'm gonna propose to you,
we are functioning up here, all right?
This is what we see, or what we know.
And I'll speak for myself here, my day-to-day practice,
that is what's going on underneath us.
All right? This is what we don't see
and what we don't know, I actually don't think we really
understand or know the scope of the problem.
And I don't propose to know it myself.
And it's not just radiology, of course,
it's germane to all of medicine.
And we, I believe are very late
to the game in addressing this problem.
And I say we, us as physicians, you know,
we're idiosyncratic with the way we're brought up to think.
It's very much independent thinkers.
You know, my practice is better than your practice.
'cause I've seen a few more cases of
this, and this is what I do.
Yeah, you may have a good way of doing it,
but I'm gonna do it my way, type of thing.
So same theme. We know this guy, right?
Interestingly, he's probably gonna go
down in history for this.
They are known knowns.
There are things we know that we know.
Sorry, these are things that we know, that we know.
It's hard to read this. There are known unknowns.
That is to say there are things we know we don't know,
but there are also unknown unknowns.
These are the things we don't know. We don't know. Okay?
Did I get that right? More or less? More or less. Okay.
So actually, I think he said something pretty profound.
He's an interesting guy.
You know, he took us
to war basically and all the rest of it.
But, you know, I think his comment here is very germane to
what we're talking about today.
Let me show you why. All right?

Case Example: Missed Liver Lesion in Breast Cancer Patient

April, 2012.
Now I'm a GI g radiologist, so I'm gonna show,
I'm not gonna show, apologies to the audience.
So I'm not gonna show astrocytomas in,
in, the brain, et cetera.
And, weird and wonderful MSK problems.
I'm gonna stick to what I know. So I, I apologize upfront.
Okay? I think anyone who's done radiology
and imaging, in fact, anyone anywhere
would probably see this guy.
All right? In segment seven
of the liver in this patient who's got breast
cancer, all right?
So hopefully we're gonna detect that, right?
And it was detected and it was interpreted as suspicious
for metastatic disease.
Why did they say that? Well, because they did their job.
They looked back in the packs.
And, here we see, I don't think I've circled it,
but there is a small lesion there corresponding to
that lesion, which has now grown in about three months time.
Now, I'm gonna ask you, is there any, any error there?
I'm, I don't, not asking for anyone to,
holler out the answer,
but by the way, if anyone does wanna stop me and disagree
and, and complain and whatever,
or even ask questions, please feel free to do so.
And I would propose for the rest of the couple of days,
I'm gonna speak for my here.
This, you know, in a way is an interactive type of,
we're all here to learn together,
and I'm here to learn it as much from you folks
as hopefully you are from all the speakers.
Okay? Well, I asked, you know, on the previous slide,
any error in that, that issue there,
but well, it turns out a different issue.
Now, it was actually missed,
in the original interpretation.
In January, 2012, the referring physician came down
who looked at it and said, what's this?
And it was re-reviewed
and reinterpreted as too small
to characterize, probably a cyst.
Okay? All right. Any error?
Well, I think we're probably, no,
no prizes there for discovering an error.
There was an interpretation error, obviously.
Alright, well, guess what?
We had this as well from September, 2011,
and the person reading it in January, 2012 said, too small
to characterize probably a cyst.
There's nothing there. All right?
You can go up and down there. There's nothing there.
So here we go from September, 2011 to here to here.
That's metastatic disease, multiple errors.
So getting back to my, my friend here,
I'm not gonna read this again 'cause I'm gonna get it wrong,
but let's take the first one.
There are no knowns.
These are the things we know that we know. Alright?
We're human. We make mistakes, okay?
On this case, we made a, we made a mistake,
and it's gonna happen with busy practices.
Even if you're fresh and awake,
we're still probably gonna make these errors.
And I'm gonna propose, I probably made an error like this,
in my, in my practice at some time,
maybe more than just a single time, okay?
Now there are, this is the next bit.
There, there, there are no unknowns.
That is to say there are things that we know we don't know.
We know we make mistakes, but we don't know. When.
I've just look what I've just said to you.
I'm sure I've made a mistake like that in the past,
but I have no clue when and where.
All right? I, I'm,
I'm making these mistakes, but I have no idea.
All right? So in this particular case,
to show you an example, radiologist number four, correctly
in, well, actually I'm gonna nuance that in a bit,
but they picked up the abnormality
and they looked back at the previous, image
and said, a suspicious for metastatic disease.
But guess what happened? Radiologist number four
didn't go back to radiologist number two
or number three to say what had happened.
So how on earth is radiologist two
or three gonna learn from this process and radio?
Neither, none of radiologists, two, three
or four Went back to radiologist number one.
Now, they didn't make an error,
but at least try and close this loop.
Now, if you really think, honestly,
and I think part of our problem is in medicine as a whole,
and I'll, maybe I should speak for myself here
'cause I don't wanna get too preachy.
We've got our head in the sand. We, we, we have no idea.
And in this instance, in my institution today,
and I'm, I will say Massachusetts General Hospital, we like
to think we're one of the best departments anywhere
and all the rest of it, this goes on every single day.
We are picking up errors and we are not closing the loop.
All right? The third one,
there are also unknown unknowns.
There are, there are things we don't know, we don't know.
And I would say we don't know
that we are making mistakes all the time,
every day in some form or another.
We have no idea of the scale of the problem.
What's the scale of the problem? No idea. Alright? We dunno.
When we miss findings, we dunno how often we miss findings.
Worse. As I said, when someone knows we made a
mistake, we still don't know.
There's no real effective loop closure. Alright?
So in this case, we missed that.
We then misinterpreted it in a way.
It was, I'm gonna show you why I think it was
misinterpreted even there.
And this is the reason, okay?
Suspicious for metastatic disease.
This is a big bugbear of mine in my department
with residents, fellows, and even my staff.
And as I'll show you tomorrow, we,
we have a certain peer review process,
which we think is pretty good.
It's by no means, optimal.
And we are not actually looking so much at the images.
We're looking at the reports and what we actually say
and how what we communicate in the reports
suspicious for metastatic disease.
We get this all the time.
How helpful is that to the referring physician?
Alright, it is metastatic disease. Alright?
What suspicious are they gonna come down?
Well, how, how certain are you of that,
that this is metastatic disease?
Are you really certain? Do I have to do another test?
What do I tell the patient?
Like, the case Mike showed, they don't even biopsy
that HCC, it shows all the classic findings of washout.
It is HCC move on.
And we, this happens all the time, even in, you know,
the best of the, you know, the org organizations,
academic teaching hospitals.
I would actually propose because the,
the folks out in private practice
and in the busier practices probably get this right more the
time because they're often much more in tune
with their referring physicians.
They're not gonna tolerate this kind of stuff.
So that's kind of a confidence error.
Lemme get this arrow out the way if I can.

Additional Case: Indeterminate Liver Lesion on Ultrasound and CT

Alright, next case, breast cancers and ultrasound.
Okay, so we got that lesion there in the,
segment two of the liver, right?
So what is it? So indeterminate CT recommended.
So this is a real case at our institution.
Alright, so we do the ct. Here's the non-contrast.
Hmm, can't really see it. Oh, there it is.
Portal venous phase and delayed phase. All right?
So I know a lot of you will know what it is, okay?
Because you're experienced
radiologist and all the rest of it.
But, you know, we, are not always on our game.
Indeterminate. MRI recommended. Now think about this.
There's a patient who's got a malignancy,
who's got a lesion in the liver.
Put yourself into the patient's shoes.
Have I got metastasis or not?
They've picked up a lesion on ultrasound. Oh dear.
Can this be cancer? Let's say it's a 43-year-old woman.
And by the way, I'll, I'll use this an example in a way.
'cause I often think we don't empathize with the patient.
A di slightly different issue.
But when I'm with residents and fellows and even staff
and I, I fall into this trap myself.
So I, I'm, you know, I'm not totally them.
Now you see an unusual case of something
and I remember, you know, a particular example is a
43-year-old woman with breast cancer
who's got three kids at home.
She's come for a CT that day
to find out if she's progressed or not.
And you come in with, she comes in with,
I haven't got the example, but she comes in
with an interesting version,
if you will, of metastatic disease.
Great case, what a wonderful case.
We're gonna show that at rounds.
Take a step back, it's a disastrous case
from the patient's point of view.
She has to go home and tell her family
that I've now got progressive metastatic breast cancer.
I can't even imagine the conversations that she has to have
with her three kids, her husband and her parents,
and her friends and her workplace, let alone herself.
So why do I bring that up?
Because this patient's anxious.
They've got something in the liver. We sent them for a ct.
Now you could argue that's not the worst thing in the
world, okay?
It's not bad practice,
but you're dragging this out from her point of view, okay?
Is this benign or malignant?
Plus as we're all too familiar, there's copays.
This costs money for patients, alright?
Plus the downtime from leaving
work and getting these studies.
Alright, indeterminate,
MRI recommended here goes the MR mri all
breast cancer, okay?
I think you folks know what it is already, right? Boom.
Even on the T two image, you know what it's gonna be?
T one in phase outer phase post gatto,
portal venous phase delayed, all right?
T two bright peripheral enhancement nodular signal drop off
on outer phase imaging, it's a manger and a fatty liver.
Now you might say, oh, come on, you know, cts fine in
that case, but we know that MRI has far more sensitivity
and specificity for characterizing a liver lesion.
That whole CT could have been avoided.
The, the anxiety, the cost, the expense,
the waste, and I would propose error.
Alright? So that's a recommendation error.

Communication Errors

Now we have communication errors.
This one was not communicated in an urgent fashion gas in
the, portal vein from, mesenteric ischemia.
Patient didn't do do well.
Here's another type of communication error.
The same issue. Here it is. Alright?
It's fuzziness around the superior mesenteric artery.
Something going on here, it was called,
but it wasn't highlighted.
That referring physician wasn't called.
It was kind of left to the system.
Here comes the patient back a few months later, alright?
Now you could argue that it was inoperable,
maybe even then the pancreatic cancer.
So it's moot, but that's not really the point, okay?
It's pretty embarrassing to have to go back
and say to the patient,
well actually it was there a few months ago.
So we have all these types of errors, misinterpretation,
confidence recommendation, communication error.
Then we have this one crass error.
And in fact, I just reviewed a, paper for j
for JACR about spelling mistakes in voice recognition.
And so this is the old type
of report, which we'll get on to a minute.
This is unstructured reporting, right?
So this could actually happen in structured reporting.
All right? Stable pneumo from a prior nipple procedure.
It's pretty embarrassing, right?
When we put that stuff out, and I know I put that stuff out
because I've seen some of it,
and you know, I'm, I'll have to say I'm embarrassed that
I don't take the time and effort consistently.
I try to consistently to read through my reports
thoroughly before they go out like this.
Because guess who's seeing it?
Your customers referring physician and what?
And they can sort of morph that and nuance it.
But worst of all, your patient, that's not good
to the patient as far as I'm concerned.
You know, we can do a better job than that.
So those are kind of the errors
that we are really familiar with.
But as, as I propose, I wanna sort of step back
and look at a bigger problem that we actually create error
in many ways in every link in this value chain.
And this value chain goes to alter this.
Everything we do in radiology,
whether it's scheduling the right test,
whether it's doing the right test,
whether it's interpreting the right test,
whether it's communicating the right, result, et cetera,
is goes towards what's called an actionable report.
In other words, everything is crystallized into that report,
which it should be succinct, clear and actionable.
What I mean by that is the referring physician can then take
the key data from that report
and perform an action, if you will, on the patient.
Whether it's another test or whether it's a treatment
or whether it's no treatment or no test.
Every, that's everything we do.
And if you create an error out
of the gate on the scheduling,
that is not an actionable report, okay?
At least it's not an optimized, actionable report.
We heard about protocols.
If you do the wrong protocol,
you're gonna undermine your ability to deliver
that action, action report.
I think you get the point in every step of this.
You can denude, if you will, the value in
that actionable report and talk about value and outcomes.
Everything from we, from from us comes down to this.
The actionable report scheduling, all right?
Protocoling procedures, reporting, distribution,
everything goes towards that.
Back to the referring physician.
I call it a desktop 'cause everything's now electronic.
And you, another word for the
desktop is the referring physician.
And I think he's right. Again, I'm gonna reemphasize it.
We just don't know what we don't know in that process.

Comparisons to Other Industries and Variation in Practices

Now imagine this was the,
the same case in banking, all right?
You go to an ATM here in Vegas and you put your ATM card in
and it gets the wrong person.
You get the wrong amount, it deducts the wrong amount.
It puts more money in. I mean, it's all over the map.
We within about three seconds, we would not tolerate this.
It would be completely unacceptable.
And the whole banking system in a way would shut down
their error rate.
I dunno what it is, but it's gonna be 0.0,
0, 0, 0 something percent.
They know how to do this. They know what the customer wants.
I can tell you what if someone takes 3000 bucks outta my,
my account without me knowing about it inappropriately.
Well, maybe we don't know about it
'cause I'm not looking at the,
statements, but, but I do.
And so we are gonna make a fuss about it.
What about these guys? This,
this plane sat in Boston for a hundred days.
JL flight, you know, the battery fire and stuff.
They have a battery fire. Boom, it's out for a hundred days.
Just think the hit to the industry on that one
to JAL in terms of its revenue, Boeing, massive loss
of revenue, et cetera, from, delay, they had
to refit, if you will,
all the batteries in all those seven, eight sevens.
These guys take it really seriously.
And you might say, yeah, they take it seriously.
'cause a plane goes down, 200 people are gonna die.
Well, in, in aggregate in the United States.
I dunno how many hundreds, if not, you know, hundreds
of people die a day unnecessarily because of our era.
All right? It's happening every day. So ERA is everywhere.
It's ubiquitous. Every step of the ROIC process
as I propose, it's really not
acceptable in other industries.
And usually we don't know about it.
Worse, when someone does wanna do something about it,
we try and resist it because I don't wanna be monitored.
I don't wanna be found out. I don't wanna be an outlier.
I don't wanna be made to look bad. All the rest of it.
And the reality is
because of this, partly we have a marked variation in
practices and I think variation in practices is a killer
for us in terms of getting back to that value and outcomes.
I know what's best. You don't kind of thing.
Alright, so let's briefly look, scheduling, this is a,
this is actually a cardiologist anyway, you read this one,
they were stenting four times the national average
and they got found out by the New York Times.
They settled actually this year. I dunno what four.
But they said every patient needs the stent,
trust me, kind of thing.
Well, they, they did a deep dive
and they looked at three out
of four patients could have been treated
with medical therapy as opposed to interventional therapy.
What about in our own profession? MRI and CT utilization.
I'm talking about appropriateness utilization here.
Now, something's odd here.
You may disagree that we should be performing imaging at the
same level of the OECD, which is the, the sort of,
organization that looks globally at,
at performance in a whole range of issues and businesses.
So we are way up there in terms of utilization number two
in terms of MRI and ct.
So why is the Netherlands down here?
I dunno what country you pick.
You know, if you were to pick something outta the air,
which you might respect and think these are good, good,
cool dudes, if you will.
And for some reason I pick the Netherlands, I've,
I've often found the Dutch extremely pleasant, warm people.
They're sensible, they're
intelligent, they know what they're doing.
So why they doing CT down at the bottom here
and we're up actually at the top.
Something odd is going on there.
Alright, who's the top, by the way? Now this is a weirdo.
If I was to ask you, what's the basket case of Europe,
no offense to Greeks in the audience,
you'd prob economically basket case.
I, I I better just, caveat with
that right now it's probably Greece,
although they're just getting out of it, right?
They're actually doing a pretty good job to get it.
But a year ago they were in the toilet.
But look where their, their utilization is.
It's way up there. What about us back home?
So in the sa in the southeast,
I've somehow they missed the, bits on the slide.
But essentially there is a 50% vari vari
variation in utilization in different parts of the country.
This part of the country being the most up here
and over here the least.
Now I know we have different populations
in different, all the rest of it.
You cannot explain a 50% variation utilization
just by those differences.
There's, there's gotta be under utilization
or over utilization somewhere.
And this is despite all of this stuff, okay? It's out there.
The a CR imaging appropriateness criteria.

Protocols and Procedures

So that's scheduling. What about protocol and procedures?
You know, I dunno how we keep up. Look at this.
It's ridiculous. 368 CT protocols at MGH.
How on earth can we choose the right protocol?
How can we devise a system to do it?
The machines keep changing,
they keep getting more sophisticated.
It's very hard for us to, standardize our protocols
and deliver the best protocol for the given indication.
The choice of modality varies.
I'll show you an example of this.
You know, do we do a follow-up CT
or an MRI look we heard from Mike again, follow up lipid,
poor adenoma, which happens to be one of my areas.
Forget about MRI, it's a contrast enhanced wash out
CT protocol, which the evidence is out there.
It's all been written and done.
Is the way to characterize a lipid,
poor adrenal adenoma?
There's too many variables
to ma ma measure, manage dose.
The radiologist Azure, you know,
well I think we should add these p pulse sequences to this
because I think we should, you know, we should do it.
This is all waste, this is all error in my opinion.
Alright, here's a who, who's this?
You know this one, someone holler it out. Marie Cur.
Marie Curie. Okay,
so we've known about this stuff since 1934
and yet only last week.
Last week now I was in a major hospital system
where the pediatric protocols are standardized
during the day for ct,
but at night, the er they do not have a set
of standardized protocols.
What is going on there? I mean, we are 2013,
we are getting close to like a hundred years here from when
she, when she died.
So radiation is still all over the map.
Granted, it's hard to keep control
of it 'cause the machines keep changing.
There's a lot of innovation going on,
but this is waste and error.

Reporting and Recommendations

What about reporting report format is variable
standard templates versus free text.
The language we use is all over the map.
You know, we do a mass versus tumor
of fluid collection versus abscess who's,
who's using the data from the electronic medical record.
The next revolution I would propose in our in, in imaging,
aside from trying to reduce error, is using all
that collateral data from electronic medical record
to deliver a far more specified actionable report.
We should be incorporating the path data, the blood data,
the biomarker data, the molecular, data, proteomics,
metabolomics, genomics, et cetera.
I would, I would propose to you
that if someone has a KRA S mutation
for colon cancer in five years, we're gonna be incorporating
that into our report
because it's possible that the morphologic appearances on CT
or the physiologic metabolic appearances on PET are gonna be
different with a KRAS mutation as opposed
to A-B-R-A-F mutation.
So we're getting into that.
Now, incidental findings, how do we handle those
and important findings, communication,
unstructured reporting.
Where's the actionable data in there?
You're asking the referring physician to go and t tra around
and to find where the data is.
This is still a big issue.
Now I'm a little bit on my hobby horse here,
but I think we do not deliver value
and frankly a, a setup for error
with our referring physicians
by bearing the findings somewhere in the report
that we are now expecting them to go and look.
And a good friend of mine and weeks ago said this to me,
I fundamentally disagree with standardized
structured reporting because it's how I as an individual
create my value by creating a report
that I feel is unique to me.
And this is someone I know some
of you in this room will know very well.
Well you may feel it's important to you,
but what about the referring physician
and the patient in the structured reporting?
Everything's laid out in a succinct matter.
The recommendations are always at the bottom of the page.
So the referring physician knows to go exactly where to go
to what are the findings, what are the recommendations?
What are the salient points? What's the actionable data?
What can I do? How can I reduce error?
How can I move on to the next step with this patient?
How can I expedite the care?
Alright, what about recommendations?
21% of cts we looked at this,
generate recommendations for further imaging.
We and mr, we're all over the map.
This is in our own department. MSK.
Now someone's recommending 33% of the time on an MR
of the lumbar spine.
Someone else, 7%. This is over hundreds of MRIs.
So you, you, you, the variability is outta the wash.
Here it comes outta the wash. So what's going on here?
Someone's recommending 33%, another 7%.
This is a study I was involved in.
Look at this for abdominal ct major discrepancies,
MIS findings, different conclusions of interval change,
presence of recommendations in terms over error.
26% in tra over error, 32%.
So about a third of the time we're making
MA we have major discrepancies
and this was for all abdominal ct
including oncology, et cetera.
And this is despite, I just pull this outta the JCR,
that we have appropriate recommendations set out
by the J-J-A-C-R.
Not in everything, but they've already done sufficient work
for us to, to radically improve our
output and our recommendations.
So here's, here's an example.
This is I do the RSS NA adrenal thing
and this is an audience response.
So this is lung cancer 2012.
What the followup recommendations
for this left adrenal lesion.
Okay, so it's, as I said, it's an audience response.
I'm not gonna ask you folks in the interest of time.
So we can do density measurements, we can do CT washouts
or follow up,
follow up CT maybe in a few months, three months, six months.
Follow up. MRI PET ct.
Remember the patient's got lung cancer or
effectively do nothing.
Alright? So if you do any of these, it's gonna cost you,
'cause this was contrast enhanced ct.
Now, so to do the density measurements, you're gonna have
to do a non-contrast ct.
So it's about 500 bucks extra radiation anxiety probably
'cause you have to bring the patient back.
CT washouts, another test, follow up ct,
another test follow up R mri.
It's about a thousand bucks. Anxiety, PET CT, 2000 bucks.
Nothing costs you zero bucks, right? No anxiety.
The most common answer was this. I'm not kidding you.
And this is in three refresher courses.
Now at RS NA by an overwhelming majority,
about 80% of people said this.
Now I, granted it was a little bit of a trick question,
but actually this happens in daily practice
and people miss this all the time.
It's a little bit like the
case I showed you right at the beginning.
It turns out there was a prior study
and it's an adenoma, no other test is needed.
You're done. Zero cost, zero anxiety,
and all the rest of you could say, well this is easy piece.
Of course I'm gonna go and look at the prior imaging
and study and all the rest of it.
I'm gonna look that it was stable
and therefore it's an adenoma.
I can tell you in practice it doesn't always happen.
And the fact that people were at the RS NA
and the refresher course, were already sort
of trying to second guess this.
I better do a pet CT because the patient's got lung cancer.
Know we know what to do here. All, all right?

Report Protocols and Turnaround Times

Report protocols.
Are we on electronic or paper?
What are report turnaround times?
I would propose this error to some degree.
When, at the same organization
that was at last week, I said,
what's your average report turnaround time?
Well, we don't know. Well what is your, benchmark?
Five days. They said, even in this day
and age, I meet the, I meet the benchmark.
They say they think that's good.
Well, that's error actually,
if you really think about it, it's waste.
You cannot, we are worthless until an actionable report
is delivered to the people who ask, ask for it.
It's meaningless. That test until
that information gets back to the referring physician.
Closing the loop in communication protocols.
What are we doing about urgent and critical findings?
So I'm gonna propose there is error all over the map.
It's much bigger.
I propose at least than we think,
and it's our job to really address this whole value chain,
and work with our colleagues in our organization.
Institutions. We can't do it all as individuals.
You know, we are mainly down on this end
reporting and communication.
But, you know, people are siloed up here in the scheduling
and, and they, they're not joining the dots.
They're not realizing that at the end of the day, this is
what we are here for this actual report.
And funny enough, there's a patient
who's coming through this whole process.
And if I was a patient going through that whole process,
I wonder what it would feel to me.
So I propose that we actually need to go, if you will,
upstream back this way,
and start owning the whole issue and the whole problem.

Peer Review Practices and Challenges

So yes, if, on that previous slide, you know,
as I said, we're down on the bottom left there.
We think often we're on reporting and communication.
Well, Johnny Kruskal, who many
of us know from the BI de Canes
2012 survey of academic radio.
So the point of this slide is even where we think we do,
we're meant to be responsible.
Okay? Peer review, at least looking at the, the major pieces
of error we at least think of when it talks to error.
This is how we do, sent it to a hundred scar members,
28 responses only.
That's, you know, these, these types of things are hard
to get the information after three reminders, only two
of the 28 have a hundred percent
faculty participation and peer review.
Only two responding departments have defined a required
number of cases to be reviewed, to claim,
to manage the collective data.
And this is gonna sound a little too preachy. Shame on us.
Frankly, from the patient's perspective,
we are making mistakes as are proposed all the time.
And we are not even really seriously trying to find out
how big the problem is.
Which gets back to my friend, with we don't know
what we don't know, et cetera.

Conclusion and Final Thoughts

So to finish up, I personally believe we have a much bigger
problem than we think there's variation all over the place,
despite the guidelines,
which has a direct impact on quality and safety.
If you think in your department, okay, I'll,
I'll speak for my own department.
If there's a recommendation to do on a test
for an incidental finding, I can still tell you today
and I'm gonna show you tomorrow by the way,
how we can minimize this and standardize the whole process.
All good people, all nationally,
internationally known radiologists,
someone will recommend a ct,
someone will recommend it in three months,
someone in 12 months, someone else might recommend an MRI,
someone, an ultrasound, someone, nothing.
This is in my own department, my own division,
let alone between hospitals.
And these differences increase the cost
and potentially affect outcomes,
which is the whole new domain
as I proposed earlier on in the healthcare dynamic.
Granted, conforming to guidelines
and best practice is a huge challenge for us.
First of all, a lot of the
evidence-based data is hard to get.
I'm not saying it's easy, but just to use that.
Some people have said that, well, you know, it's hard to get
and I don't even know if the evidence-based is appropriate
stuff and just continue as is if you will.
That's not an acceptable answer in the a lot of criticisms.
I would grant you for this article
by Brenner in 2007 on CT dosage.
And he was the first guy to really
recently at least put this on the map,
which made us all wake up about at least the referrers,
about CT dosage.
Perhaps a third of cts could be replaced
by alternative approaches or not performed at all.
Maybe a third of CT is inappropriate and it's an error.
So get back to this comment.
It's not enough to, to do your best.
You must know what to do and then do your best.
And I'm gonna, you know, really challenge ourselves to look
below the water, if you will, below the surface
and look at this issue going on right here.
So I come back to this and of course now it's a sort
of self-serving question here.
You know, are we doing very well, quite well poorly?
I'm not gonna ask people to put their hands up
and maybe I'm being too pessimistic
and, you know, a naysay all the rest of it.
But I'm gonna propose we're actually still here
despite the good efforts.
And remember, we are a biased audience.
You actually showed up to this conference, alright?
And are interested in era and are interested in changing it
and improving outcomes.
What about the 95% of your colleagues who are not here?
All right, so you think we are probably doing quite
well or maybe poorly?
Those are the majority of the answers
'cause you're actually doing something about it
and you can see the results.
Many of your colleagues don't even know what they're doing.
So with that, I'm going to finish
and we are gonna leave it to the next speakers who are going
to be talking in their particular domains
and I look forward to hearing from them.
Thank you.