Who is the better radiologist? Hint, it’s not that easy.

There’s a lot of talk about quality metrics, pay for performance, value-based care and penalties for poor outcomes.

In this regard, it’s useful to ask a basic question. What is quality? Or an even simpler question, who is the better physician?

Let’s consider two fictional radiologists: Dr. Singh and Dr. Jha.

Dr. Singh is a fast reader. Her turnaround time for reports averages 15 minutes. Her reports are brief with a paucity of differential diagnoses. The language in her reports is decisive and her reports contain very few disclaimers. She has a high specificity meaning that when she flags pathology it is very likely to be present.

The problem is her sensitivity. She is known to miss subtle features of pathology.

There’s another problem. Sometimes when reading her reports one isn’t reassured that she has looked at every organ. For example, her report of a CT scan of the abdomen once stated that “there is no appendicitis. Normal CT.” The referring physician called her wondering if she had looked at the pancreas, since he was really worried about pancreatitis not appendicitis. Dr. Singh had, but had not bothered to enlist all normal organs in the report.

Dr. Jha is not as fast a reader as Dr. Singh. His turnaround time for reports averages 45 minutes. His reports are long and verbose. He meticulously lists all organs. For example, when reporting a CT of the abdomen of a male, he routinely mentions that “there is no gross abnormalities in the seminal vesicles and prostate,” regardless of whether pathology is suspected or absence of pathology in those organs is of clinical relevance.

He presents long list of possibilities, explaining why he thinks a diagnosis is or is not. He rarely comes down on a specific diagnosis.

Dr. Jha almost never misses pathology. He picks up tiny lung cancers, subtle thyroid cancers and tiny bleeds in the brain. He has a very high sensitivity. This means that when he calls a study normal, and he very rarely does, you can be certain that the study is normal.

The problem with Dr. Jha is specificity. He often raises false alarms such as “questionable pneumonia,” “possible early appendicitis” and “subtle high density in the brain, small punctate hemorrhage not entirely excluded.”

In fact, his colleagues have jokingly named a scan that he recommends as the “Jha-Scan Redemption.” These almost always turn out to be normal.

Which radiologist is of higher quality, Dr. Singh or Dr. Jha?

If you were a patient who would you prefer read your scan, the under-calling, decisive Dr. Singh or the over-calling, painfully cautious Dr. Jha?

If you were a referring physician which report would you value more, the brief report with decisive language and a paucity of differential diagnoses or the lengthy verbose report with long lists on the differential?

If you were the payer which radiologist would you wish the hospital employed, the one who recommended fewer studies or the one who recommended more studies?

If you were a hospital administrator which radiologist would you award a higher bonus, the fast reading Singh or the slow reading Jha? This is not a slam dunk answer because the slow-reading over caller generates more billable studies.

If you were hospital’s quality and safety officer or from risk management, who would you lose more sleep over, Dr. Singh’s occasional false negatives or Dr. Jha’s frequent false positives? Note, it takes far fewer false negatives to trigger a lawsuit than false positives.

I suppose you would like hard numbers to make an informed decision. Let me throw this one to you.

For every 10,000 chest x-rays Dr. Singh reads, she misses one lung cancer. Dr. Jha does not miss a single lung cancer, but he recommends 200 CT scans of the chest for “questionable nodules” per 10,000 chest x-rays. That is 200 more than Dr. Singh. And 199/200 of these scans are normal.

I can hear the siren song of an objection. Why can’t a physician have the sensitivity of Dr. Jha and the specificity of Dr. Singh? The caution of Jha and the speed of Singh? The decisiveness of Singh and the comprehensiveness of Jha?

You think I’m committing a bifurcation fallacy by enforcing a false dichotomy. Can’t we have our specificity and eat it?

Sadly, I’m not. It is a known fact of signal theory that no matter how good one is, there is a trade-off between sensitivity and specificity. Meaning if you want fewer false negatives, (fewer missed cancers on chest x-ray), there will be more false positives (negative CT scans for questioned findings on chest x-ray).

Trade-offs are a fact of life. Yes, I know it’s very un-American to acknowledge trade-offs. And I respect the sentiment. The country did, after all, send many men to the moon.

Nevertheless, whether we like it or not trade-offs exist. And no more so than in the components that make up the amorphous terms “quality” and “value.”

Missing cancer on a chest x-ray is poor quality (missed diagnosis). Over calling a cancer on a chest x-ray which turns out to be nothing is poor quality (waste). But now you must decide which is poorer. Missed diagnosis or waste? And by how much is one poorer than the other.

That’s a trade-off. Because if you want to approach zero misses there will be more waste. And if we don’t put our cards on the table, quality and value will just be meaningless magic talk. There, I just gave Hollywood an idea for the next Shrek, in which he breaks the iron triangle of quality, access and costs and rescues U.S. health care.

If I had a missed cancer on a chest x-ray I would have wanted Dr. Jha to have read my chest x-ray. If I had no cancer then I would have wanted Dr. Singh to have read my chest x-ray. Notice the conditional tense. Conditional on knowing the outcome.

In hindsight, we all know what we want. Hindsight is just useless mental posturing. The tough proposition is putting your money where your mouth is before the event. Before you know what will happen.

This is the ex-ante ex-post dilemma. In case you want a clever term for what is patently common sense.

Dr. Singh is admired until she misses a subtle cancer on a chest x-ray. Then risk management is all over her case wondering, “Why? How? What systems must we change? What guidelines must we incorporate?”

Really? Must you ask?

Dr. Jha, on the other hand, is insidiously despised and ridiculed by everyone. All who remain unaware that he is merely a product of the zero risk culture in the bosom of which all secretly wish to hide.

The trouble with quality is not just that it is nebulous in definition and protean in scope. It can mean whatever you want it to mean on a Friday. It is that it comprises elements that are inherently contradictory.

Society, whatever that means these days, must decide what it values, what it values more and how much of what it values less is it willing to forfeit to attain what it values more.

Before you start paying physicians for performance and docking them for quality can we be precise about what these terms mean, please?

Saurabh Jha is a radiologist and can be reached on Twitter @RogueRad. This article originally appeared in The Health Care Blog.

Comments are moderated before they are published. Please read the comment policy.

  • John C. Key MD

    How about some communication? Does the radiology request call for a history, and is it filled in? Did the doctor tell her he was concerned about the pancreas, or did he just order an abdominal CT? And if he asked about the pancreas, did the radiologist ignore him and comment only on the appendix?

    In the health system where I work, techs are told not to take an x-ray if the request is not properly filled out. That would not be so bad–but the radiologists commonly seem to ignore the very clinical question that is asked.

    Communicate. I’ve found radiologists to be a lot of help if you tell them what your concerns are. Clinical correlation requested.

    • Eric Strong

      I completely agree with communicating the clinical question, provided that the radiologist commits to an interpretation before hearing the remaining details of a case. If the radiologist is biased by the clinical history, the sensitivity and specificity of the test (which are already imperfect on account of the wide variability in interpretation practices as the above posts discusses) because completely unknowable. If the sensitivity and specificity of a test is unknown, from a statistical standpoint, it can no longer be used in ruling in or out a diagnosis.

      In trickier situations or unusual findings, the history can be discussed with the radiologist after the initial interpretation. This allows the radiologist to provide a deeper understanding of what is being seen, but doesn’t sacrifice the initial objectivity necessary to keep the test a valid one.

  • doc99

    Quality metrics – they keep using that term. I do not think it means what they think it means.

  • Eric Strong

    As a hospitalist, I would take a Dr. Jha any day of the week, provided that if I disagree with the need for an additional test that was recommended (on the basis of extremely low pretest probability of the disease being considered), that I am not somehow penalized for not ordering the test.

  • guest

    The answer is that the slow, careful and thorough clinician will be scorned, until one of his patients has an unexpected adverse outcome, following which everyone involved including administrators will be very appreciative of that same thoroughness which normally attracts derision about the clinician’s “inefficiency.”

    But most of the time the speedier physician will be the more prized, by administrators if not by patients.

Most Popular