Let’s learn from No Child Left Behind: Measuring quality in healthcare

As we address the issue of quality in healthcare, there is much to be learned from other industries. I believe our current approach, though, is a dangerous one, one that won’t yield the desired results. Thus far, we’ve approached quality assurance as if healing were an industrial process, a process similar to those that yield cars, air conditioners, or even cheeseburgers.

But in an age where science, technology, and health policy are all changing at an unprecedented pace, we must ask a simple question: How should we assess quality in our industry?

The word “quality” elicits confidence, a sense we are discussing something of great value, something highly desirable and useful. Corporate quality strategies such as Six Sigma, however, can only reassure us that products have been through a predictably similar process, that they contain no deviations from their intended design. And in manufacturing, because they limit unwanted variance, these programs make sense, despite telling us nothing about a product’s intrinsic value or effectiveness, about its suitability for our particular needs, tastes, or budgets.

In healthcare, though, it is often the very variances identified as “defects” by these programs that can improve a patient’s health. Good physicians are not assembly line workers executing repetitive tasks. Patients are not cars, comparatively simple, inanimate machines lacking free will. And identifying “defects,” in the industrial sense, is challenging given our incomplete understanding of human beings’ “intended design.” So directly applying industrial quality principles to medical care is an ill-conceived, albeit well-intentioned, goal.

Healthcare quality programs must limit unwanted variance (e.g. wrong therapy, unnecessary test), while retaining desired variance (e.g. patient preference, physician judgment) and accounting for inherent variance (e.g. genetic or socioeconomic determinants of health). Failing to recognize the need for this new paradigm will lead to problems, big problems, particularly as financial and therapeutic decisions are increasingly tied to quality. In fact, we’ve already witnessed this in other industries under similar pressure to measure and improve.

In education, No Child Left Behind sought to bring qualityaccountabilityscientifically based research, and standardization to our classrooms. It also fueled discussions about tying teacher compensation to student outcomes. (Sound familiar?) But the legislation has been deemed a failure by many educators. Why? It failed to recognize that every child is unique, that even perfectly healthy, well-adjusted children learn in different ways, that teaching is more than a series of easily reproducible tasks, that standardized test scores do not accurately measure learning.

Our industry’s approach to quality has consisted of a rushed, almost frantic quest for uniformity of process and outcome, an approach not unlike the overly simplistic one used in education. And, not unlike the aforementioned emphasis on standardized testing, it has been dominated by a frenzied search for things easily measured, for data we can use to declare patient encounters “failures” or “successes.” But does that approach help us identify true quality?

If a patient has diabetes, an HbA1C of 7.9 is a failure, right? Can’t decide? Okay, let’s talk prevention. A healthy 61-year-old man hasn’t had a colonoscopy — defect? Not sure? Let’s look at outcomes, then. An 81-year-old woman with advanced dementia dies of a nosocomial infection after being admitted and intubated three weeks earlier following a massive stroke — failure? Where was the “defect”? Was it in the treatment of the dementia or the vascular event? Was it that she acquired a nosocomial infection? Was it that her suffering was extended with admission and intubation?

We can’t assess patient care based on isolated pieces of information or marginally applicable national guidelines, nor should we assume every patient interaction can be evaluated in terms of failure and success. Improving health is primarily a local endeavor that is informed, but not governed, by insights gleaned from global research. And our desire to categorize outcomes, to modify them, must be tempered by realities inherent to the human condition, realities that include free will, suffering, and death. So how should we proceed?

The practice of medicine requires compassion; programs assessing and rating it should be equally compassionate. Compassionate quality programs would strive to understand the nuances that make real patients, even those with similar conditions, unique. They would apply that knowledge to assess, rate, and improve care, not to manipulate data, label patients, or penalize physicians. They would not seek “assurance,” “control,” or “compliance” but rather realization, realization of the full potential in an inherently human system, a system limited in its ability, and in some cases, its right, to modify variables that profoundly impact the very outcomes it seeks to measure and improve.

To properly assess, rate, and improve care, a Compassionate Quality Realization Program ℠ must be:

1. Local

  • Address regional variation in genetic, demographic, or socioeconomic determinants of health.
  • Account for geographic variation in insurance coverage and the availability / cost of drugs, diagnostic tests, and other interventions.

(Without appropriate context, healthcare quality ratings mask local disparities, often correlating poor outcomes with sound care.)

2. Comprehensive

  • Assess care longitudinally.
  • Recognize both the multifactorial origins of disease and the equally multifactorial nature of healing.

(Reliance on individual disease markers / isolated outcomes dismisses the complex interrelationships within human beings / local delivery systems that impact health.)

3. Agile

  • Develop assessment / rating criteria as part of an ongoing, iterative process.
  • Seek consensus amongst all stakeholders, including patients and physicians, not just payors, as to what can and should be measured.

(Programs that aren’t agile evaluate current care using outdated guidelines and inappropriately subordinate the interests of some stakeholders.)

4. Objective

  • Evaluate outcomes fairly, without unduly rewarding or penalizing delivery systems solely based on organizational structure, size, or other indirect criteria.
  • Be reasonable and transparent with regard to funding.

(Entities assessing and rating quality must be held to the same standard of unbiased transparency as those engaged in the work they seek to assess.)

5. Practical

  • Distinguish modifiable elements of patient care from those beyond its scope.
  • Weigh scientific evidence, physician judgment, and patient behavior / preferences proportionally.

This paradigm challenges the concept of evidence-based medicine. But the profession has always demanded more than unwavering faith in transient, narrowly applicable guidelines. We must reframe the quality discussion to recognize “evidence-informed decision support” as one part of a “reason-based medicine” approach that also values physician judgment and the nature of the humanity it serves. (Scientific evidence is ephemeral and discrete; the human condition is immutable and universal. Only reason can reconcile the two, and our greatest challenge is achieving consensus in applying it.)

Compassionate Quality Realization ℠ will require hard work, patience, discipline, and significant investment. But if we want to improve outcomes and control costs, if we want quality to have real meaning in healthcare, we need to do this right. In education, we’ve unfairly labeled children “defective” and teachers “failures.” Let’s not do the same with patients and physicians, especially if we aren’t really improving anyone’s health along the way.

Luis Collar is a physician who blogs at Sapphire Equinox. He is the author of A Quiet Death.

Comments are moderated before they are published. Please read the comment policy.

  • Doug

    Great piece. It would be comforting to see some of the institutions who helped further the study of quality in medicine come forward with some credibility on behalf of the thoughts presented here (and elsewhere, similarly). Inherent in an RCA (root cause analysis) is that similar poor outcomes from similar processes have differing and novel explanations. We can’t refer to one prototypical RCA for one poor outcome. It cannot be done. And in that, there are some adverse “outcomes” that have their roots in factors outside of poor quality care. Even Deming himself said the most important things cannot be measured.

    • hawkeyemd1

      Great thought. But never gonna happen. I’m afraid history tells us if we don’t know the answer, we’ll just make one up.

      • Luis Collar, M.D.

        Agree that sometimes that’s exactly what management and policy makers seem to be doing. Hard to believe, but true.

    • Luis Collar, M.D.

      Great point. Agreed. Hopefully, we can start using a much more comprehensive approach that actually focuses on “care” and not just isolated numbers and outcomes. (And Deming also spoke of “appreciation of a system” and “knowledge of variation,” two topics that might serve as a good starting point. Avedis Donabedian also did great work in this area. Definitely worth looking at if you haven’t already.) Thanks for contributing.

  • Shirie Leng, MD

    Very good comparison of no child left behind and the current quality push in healthcare. Hadn’t thought of that.

    • Luis Collar, M.D.

      Thanks for contributing. There are several teachers in my family. Many of the things they were telling me regarding standardized testing, etc… got me thinking about the topic. So, after doing a little research, I found the similarities (in some cases) are striking.

  • hawkeyemd1

    Great article. I think we jump to conclusions about quality because the “powers that be” need something to point to. Always really enjoy your posts.

    • Luis Collar, M.D.

      Thank you very much for the comment. I appreciate the feedback.

  • southerndoc1

    “I believe our current approach, though, is a dangerous one, one that won’t yield the desired results”

    Since the only desired result is to reduce physician income, I think our current quality measurement programs are highly effective.

    • hawkeyemd1

      LOL… Coudn’t have said it better myself…

    • Luis Collar, M.D.

      You might be quite right, but it should really be about much more than that. Hopefully, we’ll take a much more thoughtful approach and start focusing on more of the things that actually make a difference in patients’ health. Thanks for contributing.

  • doc99

    You keep using that word “Quality.” I do not think it means what you think it means.

    • Luis Collar, M.D.

      Thanks for contributing. I know what quality means, both in common usage and with regard to “assurance,” “control,” “improvement,” etc… What would you suggest it means?

  • Luis Collar, M.D.

    Part of it is indeed financially motivated. But, as I said above, it really needs to be about much more than that if we really want to improve patients’ health. Thanks for your comments.

  • Luis Collar, M.D.

    Thanks for commenting. I think teachers and physicians have several things in common. Specifically, both require a realization that achieving the best results often requires a tailored approach for a specific individual. In medicine that sometimes means NOT starting a patient on a specific medication even if an algorithm calls for it. It can also mean tight glycemic control may not be the most appropriate approach for another patient. I do think we should go about measuring “quality,” but we need a much more sophisticated approach for many aspects of medical care (e.g. checklists and adherence to algorithms can and should play a role, but focusing exclusively on those approaches does little to really measure “quality” comprehensively / accurately and improve patients’ health.

    • southerndoc1

      ‘I do think we should go about measuring “quality,” ‘

      But why do you think that? What other profession – lawyers, accountants, priests – has bought into the idea that the work of its members can be measured by some sort of objective standards? Seems to me like this is something of a fool’s task, and, however it’s done, will wind up being just another distraction that takes physicians away from their patients.

      • Luis Collar, M.D.

        Good point. I have two comments, though:

        1.) Given the current push for it, it appears that these programs will be with us for the foreseeable future. It’s important, then, to try to educate corporate and other organizations running these programs as to what good medical care actually involves, and why focusing exclusively on the “easy” things to measure doesn’t really get the job done..

        2.) I do believe there is value in looking at care delivery and identifying any problems / opportunities for improvement. A solo practice does this all the time in most cases (independently) in the course of doing their jobs (whether they realize it or not), which
        is what I think you are referring to. But as corporate involvement (larger practices, hospital-owned practices, etc…) continues to increase, the issue gets a bit more complicated. In many cases, those physicians are now employees who often have little say in how different processes or algorithms are applied or changed. Who is doing the “measuring,” what is being “measured,” and how the results are interpreted is what is really important in those cases. But hospital administration, insurance companies, etc… engaged in the activity often have different motives / perspectives on “quality.” This was just an attempt to get physician perspective into the conversation in those situations.

        Your point is well taken, though. (But the lawyer and priest comparison could backfire. Just kidding.)

        • southerndoc1

          Thanks for the reply.

          Right now, we don’t know if quality can be measured, and, if it can be measured, will doing so improve care or lowers costs. What we do know, from multiple studies, is that current P4P programs do neither.

          So, as scientists, lets not sign on to something that may turn out to be counter-productive with the hope that maybe, possibly somehow we can make it a little less onerous. That’s exactly the mindset that’s saddled us with lousy EMRs, crushing MU, patient-hostile PCMHs, worthless ICD-10, etc., etc.

          Until then, why not say: show us evidence that quality measurement will produce what non-physicians claim for it. In the absence of that evidence, we adamantly oppose putting any time or money into this initiative.

  • Luis Collar, M.D.

    Agreed. That’s why I think we need to step back and actually reach real consensus. Just picking out certain disease markers or test result numbers (e.g. HbA1C), and using them to say “good doctor” or “bad doctor” is an overly simplistic approach. A patient’s HbA1C may reflect poor healthcare, but it quite often reflects genetic, socioeconomic, demographic and other differences. A wealthy patient on fifth ave in Manhattan and an unemployed patient in rural PA may both receive good care, identical care, from their physicians and yet achieve wildly different results. Things like patient choices, income level, education level, insurance coverage, local availability of specialists, and many other things will contribute to that difference. If all you do is look at the HbA1C, can you really say one doctor was good and the other less so? In some cases that may be true, but often it is not. The other aspects (unemployment, income, stress, education, diet, patient autonomy, etc..) are important as well, so they need to be accounted for to determine the role healthcare actually played in the particular “outcome.” Thanks for contributing.

  • ErnieG

    we should forgot about quality. We instead should allow patients to vote with their feet. Give citizens/patients some access to the money they are paying, and let them choose what physician/provider they want to go to. Let physicians pursue excellence.

  • Arbi Khodadadi

    I thought that patients in General had the ability to chose between doctors. I thought referring physicians had choice among specialists. Doesn’t this system allow the people who actually have skin in the game to decide what quality is?