Why popularity based payment for doctors is not the answer

Part three of the three-part series, Let’s Pay Popular People More!

Perhaps you remember Sam, the chronic inebriate whose story I shared to discuss the pitfalls of basing doctor pay on patient satisfaction surveys.

Looking at his discharge papers, I wondered who helped Sam fill his survey out, and how much their “help” affected the results.

After all, millions upon millions of dollars are already now at stake for hospitals. And individual doctors’ Medicare payments are expected to be based on their satisfaction scores, as early as the year 2015.

Surely these surveys are validated and standardized, right? Surely there is policing to prevent “helping” people fill them out? You might be surprised by the answers to those questions.

For instance, when you’re talking about something like “satisfaction,” there are some regions where patients are less forthcoming with praise (check out the difference between, say, a quiet night hospital score in California versus Alabama).

These scores also lack variability. Westby Fisher, a clinical associate professor at University of Chicago’s Pritzker School of Medicine, calculated, with the Kaiser Foundation, the mean, median and standard deviation of hospital patient satisfaction data. Nationwide, there is just a two to six percent variation. In other words, the results vary arbitrarily, but very little. By statistical standards, it’s not a very good test.

Medicare’s review of these surveys showed there is no standard and the answer options are often biased to get better results. I’ve personally been given surveys that only offered me positive answer options.

There are even widely disseminated tips for doctors on how to get a “better” result, before your pay depends on it.

No one even seems to be watching how many people fill them these surveys out. As this story in support of surveys shows, a $1.6 million project funded by the Robert Wood Johnson Foundation resulted in a popular local doctor having 84 survey results. If that same doctor has 1,000 patients on his roster (which would be considered a low-to-normal panel size), that means  92% of his patients didn’t fill out a survey after a huge investment of time and money. An 8% return rate renders any survey completely invalid. You’re supposed to throw it out.

But we don’t. Not for this one survey.

Even our national hospital patient satisfaction results show there is no mention of how many people are giving their opinion on any of these scores. Eighty percent or one percent? Ten thousand people or 50? One person 800 times? You can’t know.

As a doctor, ten people’s surveys are all you need to get your board recertification. But for me, calling and writing the Board, trying to explain that my patients don’t fill these out, have no phone, no address, are often illiterate, met total silence. “Use existing surveys from your insured patients” was the first answer I finally got. When I explained that none of them were insured, “just find ten” was the final instruction. Clearly I could game it any way I wanted.

Even more surprising is the reporting, even by professionals who should know better, of individual doctors’ results as “above average” without any mention of the total numbers reported, or the range of answers. Is an 83 score actually different, in any way, from an 87? Who knows?

It is mind-boggling that we are implementing nationwide a test with less than 10 percent variability, using non-standardized surveys, that more than 90 percent of people DON’T even bother to fill out. Or, that one person can fill out 50 times. We don’t even have a numerator or denominator.

Statistically, that’s kind of like Lake Wobegon, where everyone is above average. Even your TV ratings are held to a much higher standard. And millions upon millions of our health dollars are already being spent on this.

The repercussions of paying people based on popularity don’t just affect doctors.

Some patients will be marginalized. There are serious racial implications. Your doctor may know your answers — and how will that affect your care?

But don’t client satisfaction results tell us something important? As Kevin Pho pointed out, studies from both the Annals of Internal Medicine and the British Medical Journal did not find a strong correlation between patient satisfaction and the quality of care. As for whether tying compensation to popularity could help contain costs: it’s hard to imagine how using client satisfaction scores could do anything other than drive up costs, much less decrease them.

Don’t get me wrong.

I strongly believe that doctors should be listening to, and caring about, what their patients think and feel. In fact, I believe we may have brought this current insanity on ourselves.

How? At my own recent visit as a patient, my doctor never introduced herself. Although I had to strip down and wait half-naked in a gown, she never examined nor touched me. Her eyes never left the keyboard as she barked abrupt yes/no questions at me.

Talking to my doctor felt like I was talking to an angry, powerless, court stenographer. I spent 6 minutes with her and I could have done it over the phone and saved myself two hours. She left most of my care to a vague conglomeration of slightly confused mid-level providers and clerks, people who seemed to mingle around in the hallways, waiting for a patient to pop out of a room, to explain what just happened.

With that kind of treatment, who doesn’t want doctors to be more accountable for their patients’ satisfaction? But is it the individual doctor that creates this mess, or the system? Doctors have passively floated along as more and more of our roles became system-focused, and less patient-focused. Our patients want us to stand up for ourselves, and for them.

But popularity-based payment for doctors is not the answer — certainly not for the addicted, stigmatized, and disenfranchised patients I see. We are using our tax dollars to create a new, laughably flawed, multi-million-dollar “client satisfaction” industry to distract us from the forces that systematically caused these problems in the first place. We’re letting anyone and everyone game the system. And we’re paying people based on how well they do it.

Popularity surveys don’t put the focus back on the patient. They put the focus on what’s popular.

Jan Gurley writes for Reporting on Health, a USC Annenberg School of Journalism online community for journalists and thinkers. Her blog explores the practice of medicine on the margins of society and what we can learn from it. You can see more of her posts here.

Submit a guest post and be heard on social media’s leading physician voice.

Comments are moderated before they are published. Please read the comment policy.

  • Kathleen

    It’s not just patient surveys. Seattle Metropolitan magazine has an annual “Top Doctors” edition, reportedly based on peer rankings. Through unfortunate experience, I received a surgery proposal from their top rated mastectomy surgeon, and another from a “non-celebrity” surgeon at the University of Washington, which was both vastly more informative and included options better suited to my relative risks. After the UW proposal (and successful surgery), I now think the “top rated” surgeon should be assigned some remedial CMEs, STAT!

    The problem with patient satisfaction surveys is that they are largely based on hopeless ignorance, and the physician’s charisma. I’ve met women who were appallingly disfigured by incompetent reconstruction surgeons but who felt “gratitude” that he “tried”. I’ve met women who were completely ignorant of available alternatives until they heard about them from me (don’t even get me started on the myth of “informed consent”). Yet these women would, in utter ignorance, have rated their surgeons with top marks, and none of them had bothered to get a second opinion (citing travel and expense).

    The problem with peer satisfaction surveys is that they are hopelessly biased and often based on factors other clinical skill.

    Sorry, but “medicine by poll” is one the few things even more frightening than “medicine by bureaucracy”. And patients’ ratings are only as good as their information and understanding — “garbage in, garbage out”, as we say in IT.

    ~ Experienced Patient

  • http://drpauldorio.com Paul Dorio

    Excellent discussion. Thank you. I couldn’t agree more that paying people based on some sense of popularity is inane. I remember how popularity even played a role in who was admitted to the “honor society” in high school. College was similar though supposedly slightly more “grade-based.”

    If patient satisfaction were truly based on objective findings and determinations, then perhaps basing pay on such comments would be worth considering. But without careful systematic evaluation of the submitted data, there will be mass fraud and fabrication as each of doctor is forced to scramble to ensure that they are paid.

    • http://drpauldorio.com Paul Dorio

      Sorry: last line: “each doctor…”

  • Kathleen

    Re “systematic evaluation” — right, and the whole issue of controls is huge. How do you prevent such a system from becoming a mere marketing tool for MDs? I can see the poster on my doctor’s reception desk: “Fill out an online review at get a $25 gift certificate towards botox”.

    Yikes. Not my kind of medicine, thanks.

  • Kristin

    My background is specifically in psychological surveys–design, appropriate use, and analysis thereof.

    These surveys aren’t measuring quality of care. Not even a little bit. They can’t–the people who are filling them out don’t know what quality care would look like; they didn’t go to medical school, they have no idea what the optimal therapy for their condition would be. It’s the same problem that professors face when their students write reviews of their classes: there are some intelligent, informed students who care and who are at least somewhat qualified to judge the quality of teaching. In a survey course of 150 students, there are maybe 10-15 students in that category. The good news is that most of them will fill out the surveys.

    There are the students who are flunking. For whatever reason, they aren’t doing as well as they think they should be. This can be related to wildly impractical expectations–that the professor should personally remind them about each and every due date, for example–or to student misconduct: plagiarists who get caught write scathing vitriol about everyone involved in their punishment. It has nothing to do with whether that professor is a good professor. These students, who didn’t attend class, didn’t do the reading, are having huge personal problems, didn’t have the appropriate pre-reqs for that specific class, or are otherwise not currently competent to take the class, will also tend to write reviews. They have an investment in it: if they can denigrate the class, they feel better about their failure to perform.

    And then you have the students who don’t know, and who don’t care. This probably makes up 80% of any given class at the freshman and sophomore level. Good luck getting any sort of response from them, much less one with meaningful and useful analyses of the situation.

    What they do measure is a host of psychological variables, none of which are being controlled for. And the part where they’re accepting such low variability and not instituting validity studies of the measures is mind-blowing. Take this to an APA conference if you want to start a riot.

    It’s good that we want to measure how people feel about their medical care. Informed, engaged patients will have better outcomes. We should do it right, though, and we’re not doing it right. Even if we could and did measure all patients’ feelings perfectly, basing physician salary on it is not rational.

    • Kevin

      Thank you. Brilliant, edifying post.

  • LB MD

    The worst part of the payment notion is that Medicare payments could be impacted. Since when did elderly and disabled patients represent the norm for any doctor? Those who take Medicare are almost always non-profit with sliding scales and among the Medicare patients, there is a major divide between those who know about the scales and those who do not… Want someone living on the typical $695/month to be satisfied? Make the copay disappear. Want their survey to reflect anxiety and stress and be resultingly negative? Keep them in the dark and feeling overwhelmed as those bills keep coming. Sometimes the doc is not the one lacking in any area; the social workers who handle delicate socioeconomic matters with Medicare–Medicaid, too!!–folks are too often only taking active requests for help, not helping those silently suffering. Likewise, plenty of studies have shown that Medicare recipients have far higher rates for depression&anxiety than privately insured individuals. Mental health, though, is an area of extreme suffering in all insurance divisions; the DSM and the HICPICS codes haven’t quite met in the middle–those services are considered LEAST essential by insurers around the board; for too many reasons to mention, too many decades to tidy up, they’d much rather take the too-common PTSD in young Medicare recipients in particular (what, you mean someone sees a decade of college sucked down a drain, life in a wheelchair after being a triathalon winner, and ramen food budgets til their day of death and doesn’t always wear rose tinted glasses and grin and giggle like they’re tipsy on Champagne and indulging in chocolates and caviar?) and shove drugs–side effect rich drugs–down their throats that not long ago barely had a bottle of Advil around to swallow. The therapy options aren’t especially super, either, as most clinicians want their $120/hour average… The 80 from Medicare is just starvingly low.

    Many times, 80 IS low; paying $12/hr receptionists, $20/hr billing clerks, and either $16 for a med assistant or $35 for a clinic nurse… Split between 3-5 docs with another dozen shared between multitudes of departments, not to mention legal fees just in case someone sues you for not catching something quickly enough… That may leave a doctor’s 15mins to an hour downright ludicrous. Certainly when you DO factor in the (54% at our clinic) ones on the sliding scale, threatening to make the doctor either game a system or worse be punished for taking the tough, nearly hopeless or just plain negative and always blame gaming type patients?? It is a terrible idea. The ones who are difficult are often the worst in giving feedback, and it often isn’t feedback about US but rather about their CIRCUMSTANCES. A homeless person telling you that they feel they are equipped to tackle their diabetes is lying or drunk most often; a few are truly zen types, but as soon as you start making high marks a requirement for full payment, doctors will start referring all the low-rating clients out if only to keep making their meager $45k for 60hr weeks seeing all too many who are suffering economically. To make the doctor the victim of this system alongside the patient BEYOND their generosity in waiving payments for many… It will leave the community clinics overrun–they already, from what I am seeing, have 20 or so in the waiting room nearly all hours; it will make the doctors who are okay doing SOME charity work–okay making up for it with full insurance payment (and who ISN’T aware by now–Republican spokespeople aside–the insured ARE paying for the uninsured and THAT is why their prices are so high?!)–start that stressful “payment due at time of service, cash&Visa only” practice. Some indeed do that, and they love their $500k salaries… The rest of us can’t bear to turn away someone because they’re poor. They need us even more, lest we add uncontrolled seizures or sleep apnea or lupus or ulcerative colitis or gout… Add that to all the turmoil with being poor. I think we remember feeling the stress nonstop and not knowing if we’d get through, even if school is nothing compared to their hardships. There is a process for complaining. There’s one for praise. Let the insurers stay on the legal end only and keep HR handling the quality of care, NOT those who stand to benefit from low scores! They clearly are NOT unbiased, to keep the most basic control of such a survey…

    Let’s hope it doesn’t pass. Honestly, I’m alright making a third what I would if I didn’t take the Medicaid folks and uninsured poor ones alike… But I have my own bills to pay–even my own Drs to see, and they aren’t for a Hummer or laser surgery, either! My dwelling is modest, but who needs more than “enough” anyway? My patients’ pets are part of my rooms’ digital photo frames (the ones with however many flipped through). I would hate to take off the cats who are the sole source of companionship to several disabled(some quadrapalegic) patients just because I had to start only taking folks that could see anyone as they’ve the money to pick whomever they please. I COULD do that, but I make a conscious effort–so do we all in our practice–to balance out our wealth and our poverty to be “fair to middlin’ I s’pose.”

    Great post. I hope the right attention comes to it. Maybe more will fill out surveys for the good and realize we’ll be gone with too many bad ones–complaining they can’t get reliable transit to the clinic is still complaining. We have helped them when they complain; in Seattle, we hook them up with Hopelink if they are DSHS Medicaid clients. We certainly aren’t just docs; we are social workers as well. Are the ones upset that their insurance copay is $30 for a simple Rx refill and urinalysis or blood check of levels when they matter… Are they going to mean we have to fire a receptionist and no one can get through until an aid calls back? Will we lose the in-house infusion center and force our cancer and arthritis etc patients to endure high cost, high stress hospitals? It isn’t just a doctor getting paid with that seemingly huge bill–someone scheduled you, gave a reminder call, got your weight/BP, checked your medicine list, sent refills to your preferred pharmacy (we could have everyone haul them off themselves again), someone drew and analyzed blood, someone may have given vaccines, TB skin tests, XRays, etc… And someone regardless will be hired to clean a building we pay to lease or maintain, to be there to take complaints, to handle payroll for all the admins and techs etc, to validate parking and more… The filtered water isn’t free; the bathroom soap/tissue, also not… So much going in, so if we cut income because a DOCTOR rated low, who do we lose? Bet it isn’t the Doc since without them, it isn’t really a “medical” practice. We are already struggling and wouldn’t we LOVE to get this icky paint scheme updated? We aren’t living in luxury; the patient waiting furniture got an upgrade but ours didn’t… We honestly don’t think 99.999% would EVER think about THOSE little things. Boy, they sure would if we had them in hard grade school type chairs. What can we do? They come first but only as long as it keeps us fed (not even the sushi folks assume)…

Most Popular