Big data in health care: Are we measuring the wrong things?

We believe in data. We are scientists after all. And yet in this new era of big data is it possible we are measuring the wrong things?

Most measures of physician performance are process or intermediate outcome measures aimed at a production line model of care. Was an A1c done, did the physician use a computer to send an order to the laboratory, were antibiotics given within two hours of presentation.

Yet most physician quality is related to what Clay Christiansen calls “solution shop” work, such as diagnosing and solving undifferentiated problems, activities that require an amalgam of experience, intuition and problem solving skills, and for which good measures have not yet been developed.

In “Grading a Physician’s Value — the Misapplication of Performance Measurement” Bob Berenson and Deborah Kaye tackle the question of how one measures the value of physician work. They conclude: “The practical reality is that CMS, despite heroic efforts, cannot accurately measure any physician’s overall value, now or in the foreseeable future.”

Berenson and Kaye point to the unintended consequences of good measurement intentions, and the silence among the medical profession: “… this policy overreach could undermine the quest for higher-value health care. Yet the medical profession has been remarkably quiet as this flawed approach proceeds.”

Individual physicians are buckling under the weight of these good intentions. Seventy percent of family physicians and 80% of general internists would not choose their specialties again. The need to spend more time on the wrong work is likely part of the calculus for these physicians.  Furthermore, the half-life of existing measures can be quite short: recommendations for blood pressure in the elderly, mammogram frequency, LDL targets, and A1c goals in diabetics have each changed within the last two years.

Lynn Bentson, an internist in Corvallis, Oregon responds to the Berenson NEJM article:

I have been dinged for no mammogram on a woman with no breasts, no A1c on a long-dead patient, no follow-up lipids in a Ukrainian tourist with DKA, no stop-smoking advice on a life long non-smoker, no flu vaccine on a Christian Scientist. I used to correct all this, one patient at a time, submitting office notes, death certificates, proof of address …  because of professional pride. I can’t do that any more, because one of the new core measures is completing and signing my records within 24 hours. Its 11 pm, I need to get back to work, all the charts need today’s date.

Dr. Bentson wryly adds:

Quality counts and we, as a nation, need more access to excellent clinician/typists.

And Carol Greenlee, a Colorado endocrinologist, change agent and leader in medicine, writes about her decision to choose patient care over participation in meaningful use (MU).

I can’t tell you how defeated I feel … I have great team care going … but (with MU) my team and our standing orders will no longer be valid … already with all of the MU stuff I work until 10 p.m. on “notes.”

I have decided NOT to even try MU 2 and instead to take care of my patients … MU is turning docs more and more into clerks, so who is going to do the true medical care … with all the extra time for clerical stuff it limits time to see patients.

For physicians at the front lines of care it is increasingly difficult to meet regulations and measures that aren’t aligned with patient-centered care and efficient workflows. In my practice, an advanced model of primary care, based on teamwork, task delegation, and efficiency, we are turning in circles to avoid running afoul of compliance issues and technology barriers.

When is the HPI recorded (CPT)

My style is to greet the patient first, listen to their story, and only later turn my attention to the computer.  While I am in the room the nurse, who is also present, will begin recording elements of the HPI. But, because I haven’t yet created a time-stamp by signing in, this could look from the outside as if the HPI were recorded when the doctor wasn’t present and thus it wouldn’t count for billing.  (In team-care why isn’t it ok for the nurse to record some HPI before the doctor enters the room?) Much of our energy during each patient visit is now directed toward navigating these compliance, audit and technology issues.

Who does what among the clinical team (meaningful use)

Meaningful use (MU) is highly prescriptive about team member roles. According to MU#1 our receptionists can no longer notify the lab of the tests the doctor has requested, without incurring a financial penalty. How can we have team based care if team members aren’t allowed to do work for which they are fully capable?

How the physician is prompted to give a patient a handout. MU2 #13

Medicare is also highly prescriptive about what type of handouts are given, and how the doctor thinks to give the handouts. If a physician decides to give a handout of her own accord, this doesn’t count for Meaningful Use Stage 2 (MU2); the computer has to first provide a prompt or it won’t count.

Only 13% of office-based physicians surveyed intend to participate in Meaningful Use Stage 2. MU2 has set up rules I believe were well-intended, and sounded good around a conference table, but by the time these good ideas have gone through policy making, institutional compliance, staffing and technology and then been converted into mandatory regulations at the point of care, they can be counterproductive.

Berenson and Kaye address the low rates of participation in federal financially-incented quality measurement programs, in this case PQRS:

The meager rate of physician participation in the PQRS also suggests that something is fundamentally wrong—physicians simply do not respect the measures, and for good reason. PQRS measures reflect a vanishingly small part of professional activities in most clinical specialties.

Perhaps we measure the wrong things because the right things are so hard to define and distill into measure sets. Should we just work harder to get better measures? Or is it time to look outside of incented performance measurement for other drivers of quality? As Berenson and Kaye write:

Even if we had better measures, behavioral economists would still challenge the pay-for-performance concept, at least for professionals such as physicians and teachers, who must manage complex situations and creatively solve problems … crowds out intrinsic motivation to perform well across the board, not just on the few activities being measured.

Policy makers and payers have a dilemma: How to get the best value for the health care spend. The crucial needed step, in my view, is to recognize that incented measurement, i.e. pay for performance, isn’t the only lever to drive value within the health care system. In fact, I believe policies directed at leveraging physicians’ professionalism have a largely untapped potential for good.

The American Board of Internal Medicine Foundation’s Choosing Wisely campaign, where professional societies voluntarily create lists of care within their specialty to engage physicians and patients “in conversations to reduce overuse of tests and procedures, and support physician efforts to help patients make smart and effective care choices” is one example. Respecting Choices, an advance care planning program, based on communication integrated into patient-centered care is another.  Developing a culture of role communication, such as Relational Coordination, within an organization is a third.

Finding the right balance between regulation, guidelines and empowered professionalism will never be easy, and there may always be a dynamic tension between those who develop measures and those who are measured. Understandably, individuals who have invested their careers in measurement can come to believe that most aspects of patient care can and should be measured. More problematically, such professionals may also come to believe that physicians cannot be trusted to provide appropriate care unless prodded, chided, rewarded or punished by performance measurement. This, I believe, is a counter-productive view.

I have seen how an emphasis on data can unite an organization and create a strong culture of improvement. But I have also seen it contribute to an under-current of fear and cynicism: Someone is always watching me. The misuse of data can contribute to a leadership approach of “command and control” rather than one of “discover, innovate and spread.” Physicians, nurses and other health care professionals are being viewed less as knights to be empowered in their service of patients, and rather more as knaves not to be trusted, or pawns to be manipulated. An environment that increasingly approaches physicians as knaves, will likely promote more knave-like behavior.

If we can re-envision the work for which physicians have been trained, turning away from an production line mentality, where the doctor is a robotic worker on the line, who must be made to process through a series of drop-down boxes to record every element of care and justify every variation from routine, and instead develop a “solution shop” mentality, where the physician is supported with data, but not strangled by it, and where the physician can use that data, along with guidelines and professional recommendations, to craft a plan of management of the unique person in their care, I believe we will travel further down the road toward a stronger, more affordable health care system.

Christine Sinsky is an internal medicine physician who blogs at Sinsky Healthcare Innovations

Comments are moderated before they are published. Please read the comment policy.

  • LeoHolmMD

    “Yet the medical profession has been remarkably quiet as this flawed approach proceeds.”
    The medical profession has not been quiet, they have been silenced. If you are referring to the paid clowns who claim to represent physicians, that is another matter.

    • guest

      We have all been socialized to be “respectful” and “humble,” and to not complain.

      These qualities are being used against us, with great effect.

    • southerndoc1

      It should be noted that the author of the original post holds positions with both the AMA and the ABIM.

  • SteveCaley

    “The practical reality is that CMS, despite heroic efforts, cannot accurately measure any physician’s overall value, now or in the foreseeable future.”
    In microbiology, evolution and business, you get what you select for.
    The assignment of easily-collected but irrelevant data points to stand in for intelligent comprehension of a complex system drives the system off the rails.
    If they had used the measurements of progress used in American Medicine on the Manhattan Project, they’d still be working on the Bomb.
    Big Data is not only annoying – it is an active force of destruction of American Medicine. but we do not care, somehow. By now, American Medicine has only a trivial percentage of those who apply value to the care of the patient. The remainder are hitch-hiking moochers who struggle mightily to pretend that they are involved. The clinical statistician in a friend’s practice is called a Data Clinician.
    The only endpoint to American Medicine is once it gets rid of the actual patient care, the rest will go smoothly. Good luck with that! I am trying as hard as I can to clear the blast area as fast as I can.

  • Thomas D Guastavino

    You are under the mistaken impression that

  • disqus_McUkQK6a8K

    I agree that we look like we are silent Who do we tell? Our professional groups write up bland objections No teeth behind them There is noone in this country who advocates for primary care.

  • Lucia Sommers

    Your post, “Big data in health care: Are we measuring the wrong things?” makes excellent points:
    - the intangibles of health care quality (“solution shop” work, such as diagnosing and solving undifferentiated problems, activities that require an amalgam of experience, intuition and problem solving skills, and for which good measureshave not yet been developed)
    - how it’s “time to look outside of incented performance measurement for other drivers of quality”
    - the unexplored value of‘ leveraging physicians’professionalism’ which relates to your apt metaphor about approaching ‘as knights to be empowered in their service of patients, and rather more as knaves not to betrusted, or pawns to be manipulated’

    Your suggestion for a “’solution shop’ mentality …where the physician is supported with data, but not strangled by it, and where the physician can use that data,along with guidelines and professional recommendations, to craft a plan of management of the unique person in their care” is exactly what my colleagues and I have counted on to
    create the practice-based venues for ‘solution shop work’ in the San Francisco Bay Area.

    Since 2005, as part of the University of California, San Francisco CME ‘Practice Inquiry’ program, once or twice monthly, PCPs meet in their offices and clinics to present real-time individual patients that perplex or frustrate them. TOGETHER they search the literature for relevant evidence, examine decision making heuristics to reformulate strategies, return to the patient with new ideas, and follow the patient over time to recalibrate judgment and learn. This approach and other small group learning methods, are described in our new book,“Clinical Uncertainty in Primary Care: The challenge of collaborative engagement.” http://www.springer.com/medicine/book/978-1-4614-6811-0) Constituting primary care versions of ‘time-outs,’ would these colleague collaborations meet your definition of re-envisioning an environment that appreciates physicians as knights and not knaves?