As a health care writer and policy analyst, I frequently encounter the term “risk” in discussions of medical issues. I also frequently see the term grossly misused in both the popular press and medical literature. Nowhere is this more evident than in the 2016 and 2022 CDC Guidelines for the prescription of opioids in the treatment of pain.
In science, the term “incidence” is a measure of the likelihood that a defined outcome will occur. It answers the question, “How often does “A” happen?” Sometimes incidence is also applied to the co-occurrence of two different events.
“Risk” is a very different beast. We say that if “A” happens, then “B” will follow or co-occur with some knowable likelihood. We assume — but very often do not actually demonstrate – that there is a relationship between A and B.
Analysts very often forget the most basic principle of statistics and probability: Correlation is not cause
Simply because two events may occur together or in sequence doesn’t mean that one caused the other. The events might have no relationship at all, or they might be related to a third and unexamined cause. To assess cause and effect, we must first validate plausible mechanisms of action. Much of the medical literature fails to address this process.
“Risk” is applied primarily to outcomes that are considered to be “Bad Things.” Clinicians would never write about the “risk” of live births among a population of healthy women. However,” risk” appears often when discussing stillbirths. Clinicians assume stillbirths are a knowable “Bad Thing” with knowable causes. Having characterized such causes, doctors may then profile their patients to recognize people with additional oversight or support needed to avoid Bad Things.
It is remarkable that the term “risk” appears hundreds of times, and “incidence” hardly at all in the 2022 revised CDC opioid guidelines. This is particularly true for the “risk” of addiction or overdose in patients prescribed opioid pain relievers. I propose that this orientation comprises an unrecognized “expectation bias.” When reviewing the medical literature, the CDC guideline writers saw what they expected to see — and excluded what they did not expect. This might also be called “cherry picking.”
In both medical and popular literature, it is frequently asserted that the U.S. crisis in opioid addiction and death was “caused” by clinicians over-prescribing opioids to their patients. However, we now know that this assertion is untrue – despite ongoing CDC claims of a cause-and-effect relationship. We also know that opioid addiction in medical patients is both rare and unpredictable.
Although prescribing rates and opioid mortality rose steadily from 1990 to 2010, these trends suddenly separated in 2010. Prescribing dropped by 55 percent, even as deaths involving opioids tripled. It is clear that opioid-related mortality, at least as far back as 2012, has been driven mostly by illegal street drugs, not legitimate prescribing. U.S. CDC should have known this reality even in 2016.
We should at least suspect a similar lack of cause and effect before 2010. Even when abused by “recreational” users or persons with addiction, opioids dispensed by a pharmacy are significantly safer than street drugs contaminated with illegal fentanyl. The easy availability of prescriptions diverted to street resellers by pill mills may have contributed to addiction. But addicts are now dying from heroin, methamphetamines, and imported fentanyl. Some published studies contradicting this reality have suffered from discrepancies in study collection methods. Even well-constructed studies may fail to document wide variations in patient opioid metabolism or adverse outcomes attributable to under-treated pain.
Another problem in statistics: Misuse of “odds ratios”
Some writers say, “Event A is 15 times more likely in people who do thus-and-so than in people who do not.” Much goes unvoiced in such phrasing. Event A might occur less than 0.1 percent of the time — even among people who do thus-and-so, and much less often in people who DON’T do thus-and-so. Odds ratios are useless unless one also knows the underlying incidence; they may even be deliberately misleading.
How do analysts or doctors recognize outcomes that occur less than 1 percent of the time? More fundamentally, how do they recognize “confounding” factors influencing outcomes but missing from patient medical records?
Retrospective studies of medical records may suffer from significant “confounding factors.” This is particularly important when dealing with small cohorts in a large population. Medical records analyzed at a distance from the clinical encounter are only as accurate as busy clinicians, and knowledgeable (or honest) patients can make them.
When data analysts seek to “isolate for” some factors to the exclusion of others, how do they address the natural variability among practitioners themselves? How do they isolate for expectation bias or the Hawthorn effect (following from positive face-to-face relationships between patient and clinician)?
Robert S. Mendelson, the author of Confessions of a Medical Heretic, suggested over 40 years ago that perhaps half of everything a clinician learns in medical school is obsolete within five years after graduation. That ratio seems unlikely to have changed in an era of rapid technology change. Thus, we now need to ask:
“How do we isolate for variations in clinician experience and knowledge? When is this variation important in retrospective studies?”
“When are individual patients comparable, and when are they not? How do we address variations in the severity or progression of underlying medical disorders?”
“How do we recognize clinicians who protect themselves from license sanctions in a hostile regulatory environment by misdiagnosing patients with “opioid use disorder” and unilaterally tapering or discharging them?”
Clinicians and policy makers need to cultivate an attitude of polite skepticism concerning the conclusions of any “study” where reported outcomes affect fewer than 5 percent of the relevant patient population. Multiple unrecognized confounds may overshadow any generalizations that we attempt.
Small cohort studies may now comprise almost the entirety of the medical literature on addiction, drug-related mortality, and adverse outcomes of opioid therapy. Failure to recognize this reality is a major and possibly fatal source of bias in the U.S. CDC opioid guidelines.
We must do better.
Richard A. Lawhern is a patient advocate.
