From denial to acceptance: Getting doctors behind performance data

As the Center for Medicare & Medicaid Services prepares to enhance its Physician Compare website to include quality of care information, practices across the country are getting ready to share data on the care of their Medicare patients or face penalties come 2015.

Publicly reporting performance measurement data is something we have been doing for the past seven years with 16 communities across the country as part of the Robert Wood Johnson Foundation’s Aligning Forces for Quality initiative. These communities publish reports about the quality of care physicians provide so that the information can be used by everyone who gets, gives, and pays for care.

What we have seen in our work with physicians as we report data on their performance is something akin to watching someone go through the stages of grief. Physicians are initially a little defensive. They feel that their decisions when treating patients are nuanced, and can’t be judged by objective measures. Then, they deny the results, and often question the validity of the data itself. But gradually there is acceptance. They come to see that the data from measuring and reporting is a useful tool that can be used to help improve the care they provide day in and day out to patients. It is at that point when we see real potential to improve the quality of care.

The road to acceptance is not an easy one, and physicians tend to have conflicting perspectives on publicly available performance information. Some of us see value in learning how practices compare to national or local benchmarks to see where they can make improvements. But there is also concern whether the information that is reported is an accurate portrayal of the medical practice, or if the data is meaningful to patients.

We’ve learned that motivating and engaging physicians are key factors in getting to the acceptance stage. In Wisconsin, when the Wisconsin Collaborative for Healthcare Quality began reporting performance data about its member practices, they were motivated to raise their game so they could be identified as a top performer and stay off the bottom rungs of the ladder. Practices improved performance on every one of the 14 reported ambulatory care measures, with the greatest improvements in diabetes-related measures.

The Health Collaborative in Cincinnati worked with its physician leadership group to engage physicians around data on diabetes care by providing group learning sessions and one-on-one coaching to the physicians. This resulted in a 7 percent increase in the number of primary care providers meeting the average rate of five key diabetes measures in the first year of the program.

In Cleveland, when Better Health Greater Cleveland reviewed its data on pneumonia vaccination among diabetes patients, it noticed that the MetroHealth System had the best performance. The alliance asked MetroHealth to share best practices with other systems so they could match the interventions. Vaccination rates in practices across northeast Ohio climbed from 70 to 82 percent in three years.

These improvements would have not been possible without transparent performance data. Public reporting of performance is not about playing gotcha. Rather, it is data that can make a difference. As we prepare for the Physician Compare enhancements, we hope the lessons we’ve learned can be brought to scale and that doctors will see the true value in performance measurement and reporting.

Robert Graham is national program director, Aligning Forces for Quality.

Comments are moderated before they are published. Please read the comment policy.

  • guest

    In other news, our medical school graduates are staying away from primary care in droves…

    It is certainly true that you can take a group of people who don’t have the option of changing careers, and use shame to force compliance with a system of attending to “work performance” metrics of dubious value.

    But those people are not only retiring early, they are telling their children not to go into medicine. A recent meeting of a mother-son volunteer organization I belong to included a recognition of graduating seniors, in which each of them spoke in turn about their favorite charity, where they were headed to college in the fall, and what they planned to major in. Out of a group of 25 community-minded, bright and accomplished young men, there was exactly one who planned a career in medicine.

    Also, our patients do not come to us to have their metrics buffed, they come to us to be cared for. A system that forces physicians to attend to metrics that are of no value to the patient himself is destined to produce dissatisfied patients.

    P.S. Not a primary care provider here, just an alarmed observer.

    • Dr. Drake Ramoray

      The only people who favor patient satisfaction surveys and pay for performance are government beuracrats, insurance companies (these two in order to reduce payments to physicians), and consulting firms (to have a relatlively meaningless measure to justify their existence.)

      I don’t think “work performance” metrics aren’t of dubious value, I think they are bad for patients and doctors. The easiest way for doctors to improve their “performance scores” is to move to an affluent suburb.

      • doc99

        If only government were paid for performance …

        • Lisa

          Think of the arguments over metrics that would ensue if that were the case….

          • guest

            Forget about the government, I would just like to see hospital administrators paid for performance. I wrote an order set for my department over ONE YEAR ago and am still waiting for the EPIC team to build it. I would be fired if I kept anyone waiting one year for anything…

          • Lisa

            Well, if one of the administrator’s metrics is profit for the parent corporation, the administrators might do rather well . . .

    • buzzkillersmith

      Bingo. Primary care is a death of a thousand cuts, some deeper than others. Only idiots like me go into it.

      I shake my head at all this.

  • Arby

    What are patients to make of all this?

    I hear this is all for my benefit, yet I have often wondered about my automony in these situations. I don’t believe in the science behind routinue mammagrams and breast cancer doesn’t run in my family. There is some science to support me on this. The same goes for a number of things from statins to high blood pressure. The science changes, yet if I choose to go against whatever the current wisdom is, I am labled non-compliant, plus my physician suffers.

    To be honest, if I am going to see this kind of social pressure, I would just as soon avoid getting medicial care unless I have an emergency. I suppose that is what some want after all. I pay my insurance bill but never use the services.

    • Patient Kit

      Paying for your health insurance but never using it for any medical care that would trigger a claim does seem to be one of the major goals of the insurance companies. High deductibles and copays and a general strategy of discouragement will definitely keep some patients from actually seeking medical care and using their insurance.

      • doc99

        Ah! You have uncovered the dirty secret about those Exchange insurance plans. They are the unholy union of an HMO and a High Deductible Plan – the worst of all worlds.

        • Patient Kit

          It’s not so secret. The cat’s kinda out of the bag.

    • Peter Elias

      There should be incentives for the clinician to explain the options and answer your questions, not rewards (bribes) or punishments designed to drive you toward an outcome selected by the clinician’s employer or your insurer or CMS. Paying me less if you decline a mammogram or pneumovax is unethical, in my opinion. Common, but unethical.

      • Arby

        Unethical. True and part of the reason I wrote my comment. What makes an incentive? My gratitude? I would thank my PCP if he took the time to explain options, yet the time is limited and I’d really rather him spend it on my pressing issues.

        I’m not a difficult patient, rather I am very laid back. The majority of the time I defer to my physicians for their knowledge and experience; I’ll pretty much try anything they suggest and I am compliant with it unless I find it not working or I have unbearable side effects. Recently, I’ve had two endoscopies without sedation, just xylocaine spray. In order to do this I had to be able to trust.

        But, I am already in an awkward position because of my personal belief that I am responsible for my own health. It seems to them make many uncomfortable. Still, I don’t know if they are concerned about me or their performance rating. And, that is why this kind of thing makes my situation worse, and I don’t need the additional pressure on my relationships with them

  • PrimaryCareDoc

    Dr. Graham- your use of the word “acceptance” requires some parsing. “Acceptance” does not necessarily mean “embracing” or “agreeing.”

    I “accept” this kind of metric bs the way I would “accept” that if someone is holding a gun to my head, I need to give up my wallet.

    • Patient Kit

      Just hearing the words Dr Graham, has me flashing on Burt Lancaster’s Doc “Moonlight” Graham in Field of Dreams. Sigh. I bet he was the kind of primary care doc that everyone around here is always talking about. And he was real, not fictional.

    • guest

      What about that other word he used…”engagement?” Where do you stand on engagement?

  • Patient Kit

    Do any of these data sets include data about how many risky, complex cases doctors take on? Because data can be pretty meaningless without proper context. Without context, it’s easy to manipulate and spin data to mean whatever the manipulators want it to mean. A critical thinking patient, when confronted with a doctor with perfect data, will wonder whether that doctor only takes the easiest cases.

    Also, using the stages of grief as an analogy really begs the question of what we patients and doctors are mourning the loss of. Data manipulators would be better off not reminding us of what we’ve lost and what we are on the verge of losing.

    • buzzkillersmith

      Excellent point. No, risky complex cases are not much emphasized. Thinking is hard for hamsters. Better for a physician with 11 years education after high school to make sure the boxes get checked and the charts get buffed.

      They figure they’ll let the subspecialists make the diagnoses.

      • Patient Kit

        All this talk of hamsters always makes me think of Hamsterdam in The Wire (one of the best TV shows ever made) and how much I wish they’d done a sixth season that focused on our healthcare system.

  • LeoHolmMD

    Congrats on all the diabetic metrics. While you were paying attention to those, diabetes blew past 23 million patients in the US, and is still rising. The costs and complications continue to rise, perhaps even fueled by metrics like the ones you mention. Nice performance.

    • Dr. Drake Ramoray

      Too bad Endo’s don’t want to take care of the diabetics anymore with all of their “performance measures” either.

  • Patient Kit

    I used to have a “The beatings will continue until morale improves” t-shirt. I wonder if it’s in a closet somewhere. I should dig it out.

  • LeoHolmMD

    Good point. A lot of the improvements noted on performance may fail to produce a single measurable outcome since they are based on surrogate endpoints. Much is wasted on meeting metrics on people with trivial amounts of disease or risk. The amount of energy it takes to budge a reasonably performing provider through that extra amount of metric meeting may even obliterate the cost effective nature of the measure to begin with.

  • NewMexicoRam

    Dr. Graham is a “national program director.” In other words he doesn’t get his hands dirty with patient care.
    That’s really all I need to know.

  • azmd

    Another unintended consequence of pay-for-performance schemes is how the data colection aspect affects the patient. A relative who works as a volunteer counselor in a family planning clinic recently pointed out to me that clients coming to her for intake are increasingly annoyed by the mandatory data collection that is shoehorned into their intake appointment. She now has to ask her new clients not only if they smoke (relevant) but if they roll their own cigarettes (relevant probably to some public health initiative but hardly to the teenager coping with an unplanned pregnancy).

    My relative is a volunteer worker i.e. not reacting to income penalties, nor to public shaming but purely to the impact on her patients. She sees that her patients are capable of figuring out when their time with their practitioner is being used not for their benefit, but for the purposes of data collection, and she is seeing it adversely affecting the patient-provider relationship.

    If this phenomenon is pronouced enough that a relatively sheltered volunteer can notice its impact on her clients, and bring it up with me as a concern that she has with our healthcare system, things are bad indeed.

    Of course since performance data initiatives are largely driven by government regulators and physicians who have not enjoyed clinical practice enough to stick with it, I suppose one wouldn’t expect the patient’s actual care to be an important consideration.

  • Peter Elias

    The improvements Dr. Graham points to are improvements in the metrics. They may – or may not – be accompanied by actual improvements in outcomes. I can improve my BP metrics with beta blockers without preventing heart attacks or strokes. I can lower the A1c to < 7 and cause my elderly patient to die of complications from a hip repair after she falls because of hypoglycemia.

    Performance measures are only useful to the degree that the performance being measured is useful.

    • PrimaryCareDoc

      Yup. I had a little old lady who had an MI. She was in her 90s. No history of hypertension. Of course, she was discharged on a beta blocker and an ACE inhibitor, because you have to.

      Really, you have to.

      If you try to discharge a cardiac patient without these meds, you need to write up a whole spiel about why you’re not following the “standard of care” so that you can defend it to the bean counters.

      Of course, the patient was re-admitted a day or two later with multiple syncopal episodes from her hypotension.

      This is what happens when we let policy dictate medical care.

      • SteveCaley

        In the modern age, there is no responsibility; only deniability. “I cared properly for the patient; however, she died.” People who can believe Newspeak, your time has come.

    • SteveCaley

      And performance data is selected with a bias towards quantifiability, enumerability and proportionate comparison; relevance notwithstanding. Those who know nothing about statistics believe that a number can be placed on everything, and that number is relevant and comparable. People who are ignorant of the Central Limit Theorem blandly discuss outliers as abnormalities.

      When you worship things that you can’t understand, that’s superstition, according to Stevie Wonder; and he’s right.

      “When you believe in things that you don’t understand
      Then you suffer…
      Superstition ain’t the way, no, no, no…

      Stevie Wonder – Superstition Lyrics”

  • pmanner

    See: Hawthorne Effect, the.

    Sorry, that 7% increase in Cincinnati is close to statistical noise. And we have no way of knowing what other issues were ignored while PCPs were trying to match these arbitrarily selected metrics.

    Ditto for vaccination. It took three years to go from 70 to 82% rates of vaccination, in a group of patients and providers who were probably harangued daily that this was vital to the security of the commonweal? Not impressive.

    How about some real studies? Parker, 2012: “Some studies have found positive associations between stroke metric compliance and improved patient-centered outcomes. However, high-quality studies are lacking and several methodological difficulties make the interpretation of the reported associations challenging. Information on the impact of public reporting of stroke quality metric data is extremely limited. Legitimate questions remain as to whether public reporting of stroke metrics is accurate, effective, or has the potential for unintended consequences.”

    Renzi, 2012: “Reporting of performance data may have a positive but limited impact on quality improvement.”

    Rinke, 2014: “State-Mandated Hospital Infection Reporting Is Not Associated With Decreased Pediatric Health Care-Associated Infections”

    Safavi, 2014: “Publicly reported quality measures for surgical site infection prevention do not distinguish the majority of hospitals that patients are likely to choose from when selecting a surgical provider.”

    • guest

      On a more positive note, all of these quality/performance initiatives do constitute a Full Employment Program for Physicians Who Don’t Like Clinical Practice…

  • Peter Elias

    Well, it’s easier to look for the lost car keys under the lamp post where the light is good, rather than in the stairwell where they were dropped. Proxies are much easier than outcomes, and people forget that their are associated with but not causative of outcomes. Changing the map will not change the territory.

    I’m all for measuring, but not just for the sake of measuring. It is important to measure the things that make sense.

  • Dr. Drake Ramoray

    I forgot about delusional medical societies who are planning a suicide mission for their profession because they think NP’s won’t practice independently and thus take the job of the doctors whom they supposedly represent. (Hint: NPs already do in many states).
    I stand corrected and appreciate your addition.

Most Popular