A recent study published in Science, one of the world’s leading academic journals, found that a predictive health care algorithm discriminated against black patients.
The tool, created by Optum, was designed to identify high-risk patients with untreated chronic diseases, thereby helping administrators re-distribute medical resources to those who’d benefit the most. But there was a glitch in the algorithm, according to researchers. Rather than ranking the needs of patients based on the severity or complexity of their illnesses, the algorithm relied on a surrogate measure: the cost of each patient’s past treatments. The problem with that is black patients receive less care than white patients with the same disease burden and, therefore, account for less medical spending on average.
When the researchers went back and re-ranked patients by their illnesses (rather than the cost of care), the percentage of black patients who should have been enrolled in specialized care programs jumped from 17.7% to 46.5%.
Since the error was discovered, Optum’s now-notorious algorithm has become the subject of intense media scrutiny, resulting in widespread outrage throughout the scientific community. News coverage of the controversy leaves no doubt about who (or, rather, what) is to blame.
“US Hospital Algorithm Discriminates Against Black Patients,” reads a British Medical Journal headline. “Racial Bias Found In A Major Healthcare Risk Algorithm,” says Scientific American. “Millions Of Black People Affected By Racial Bias In Healthcare Algorithms,” notes Nature.
Blame it on the a-a-a-a-a-algorithm
To understand what the media and scientists got wrong, ask yourself: How did an algorithm discriminate against black patients? The answer is simple. It didn’t. The technology didn’t discriminate against black people, doctors did.
The research published in Science falsely insinuated that racial bias occurred when the computer programmers started punching in zeros and ones. In fact, the bias in question took place long before that.
For decades, American physicians have discriminated against patients of color, providing them with sub-optimal care. Want proof? You can read about racial disparities in breast reconstruction, or how black and Hispanic patients are less likely to receive pain medications, or any of the dozens of clinical studies that prove racial and ethnic minorities receive poorer care than whites.
If physicians hadn’t discriminated against black patients, the algorithm would not have demonstrated bias. That’s what’s wrong with the headlines and analysis surrounding this story. Racial discrimination wasn’t the byproduct of a computer algorithm. It was the result of physicians providing black patients with insufficient and inequitable treatment.
Racial bias has long been a reality of American medicine. And it’s not the only form of discrimination that’s holding back U.S. health care.
How technophobia in health care harms us
As the Science article shows, humans have a tendency to blame technology when it fails to perform as desired or expected. The truth is, however, Optum’s algorithm performed its function perfectly.
To understand why we blame technology when human performance is at fault, consider a study from the University of Michigan, which examined a “new way to test self-driving cars” in order to improve vehicle safety.
For context, there are roughly 40,000 traffic deaths on U.S. roadways each year. Given the high death toll, you might think drivers would be eager to embrace driverless vehicles. But, according to University of Michigan researchers, “For consumers to accept driverless vehicles … tests will need to prove with 80% confidence that they’re 90% safer than human drivers.”
To put that finding in perspective, humans would not embrace self-driving cars if they only prevented 20,000 or 30,000 deaths annually. To be considered safe enough to replace humans, driverless cars would need to save more than 35,000 lives each year, according to consumers.
It’s Complicated: America’s Relationship With New Technologies
When it comes to new technologies, Americans are of two minds—or, more precisely, two sets of emotions:
1. Appreciation. Technologies that are either (a) helpful like Siri or (b) charmingly anthropomorphic like Anki Cozmo tend to elicit positive feelings, including empathy, appreciation, and even affection. That’s because we perceive these forms of technology as “on our side” and “one of us.” And because most consumer technologies fall into this category, 85% of Americans believe digital technology is a good thing for our country.
2. Aggression. When machines are perceived as “others” or “threatening,” we respond with fear and anxiety. The now-familiar motif of “man vs. machine” has been around since the industrial revolution, dating back to the Luddite Rebellion of the early 1800s, wherein English textile workers destroyed machines that threatened to automate their jobs. For modern-day interpretations of this phobia, see 2001: A Space Odyssey or the Terminator franchise or West World.
Where do these fears come from? Psychologically, it isn’t the threat of robotic annihilation that scares us. What we’re actually afraid of is the loss of control to an entity we don’t understand.
Fear of the unknown or of “others” can be a powerful motivator. It helps explain why some doctors discriminate against certain groups of patients and, likewise, why news outlets seem obsessed with technology’s failures.
In health care, an example of our pro-human bias manifests in medical errors, which kill 200,000 Americans each year. Human mistakes in health care are the third-leading cause of death (behind heart disease and cancer). Given the magnitude of the problem, you’d think these errors would dominate news headlines, but they don’t. You know what has sparked media frenzies and rigorous debate at medical meetings, despite being incredibly rare? Medical problems caused by artificial intelligence.
The only thing to fear is an unwillingness to change
Compared to the best radiologists, today’s AI solutions can identify lung and breast cancer and diagnose pneumonia on chest X-rays just as accurately or even better.
That’s because machine-learning software can compare thousands of normal studies against thousands of abnormal ones, spotting hundreds of minor differences in mere seconds. By contrast, doctors have to rely on heuristics to reach their conclusions, applying a few mental shortcuts to make determinations and reach a diagnosis.
AI applications can’t yet make perfect diagnoses from visual images. But in cases where they’re more accurate than radiologists, why not replace people with artificial intelligence, immediately? After all, if patients were asked to decide between a radiologist who was accurate 80% of the time and one who was right 85%, the choice would be obvious. AI isn’t more prevalent in medicine now (and likely won’t be for many years) because humans are biased against technologies they don’t fully understand.
Today’s health-tech experts are reluctant to suggest that computers will someday replace humans, arguing (or, perhaps, hoping) that machines will merely play a supporting role. The fact is, IT solutions can lead to lower health care costs and improved quality. But for that to happen, we must be willing and able to embrace a more tech-focused future (including a future in which these technologies replace some physicians).
To prepare the profession for the inevitable, medical schools will have to instruct students in the development and use of advanced IT. The National Institutes of Health (NIH) and other regulatory bodies will need to fund research that compares new technologies to the current physician performance (without holding technology to some theoretical ideal of robotic perfection). Finally, insurance companies must pay for high-tech services, just as they do for care delivered by humans.
Going forward, both physicians and patients need to stop viewing technological tools as threats. The health of our nation and affordability of our health care depend on it.
Robert Pearl is a physician and CEO, Permanente Medical Groups. He is the author of Mistreated: Why We Think We’re Getting Good Health Care–And Why We’re Usually Wrong and can be reached on Twitter @RobertPearlMD. This article originally appeared in Forbes.
Image credit: Shutterstock.com
