Do satisfaction scores really measure quality care?

Does patient satisfaction matter? The answer is a resounding “yes.”

In fact, as the director of an emergency department, feedback from dissatisfied patients has provided both me and our group with an early warning about several physicians who were not performing up to our standards. Studies show a clear correlation between decreased patient satisfaction and increased medical malpractice risk, so meeting our patients’ needs is not only in their best interests, but it is also in our best interests.


There is a huge difference between a qualitative phone call and a quantitative survey, though. When we attempt to quantify and compare patient satisfaction scores, we take a good thing and pervert it. Patient satisfaction is an important part of medical care, but patient satisfaction rankings harm us and harm our patients.

Not too long ago, I opened a three pound bag of M&Ms, grabbed a handful, showed them to my ten year old son, and asked him how many blue M&Ms were left in the bag. He furrowed his brows at me and said “you can’t tell just from looking at a handful.” My son understands the concept, yet satisfaction survey companies apparently don’t. They routinely perform advanced statistical calculations on the results of a handful of surveys taken from many thousands of patient visits. When hospitals or contract management groups then tie physician compensation or even physician employment to monthly “numbers” that don’t come close to meeting statistical significance, they incentivize and penalize physicians for what amount to random events. A lack of statistical validity is only part of the problem, though. The larger issue is that satisfaction rankings grade physicians on inappropriate metrics.

Bedside manner is an important part of patient care, but when patients come to an emergency department with an emergency condition, first and foremost they want a competent physician who delivers quality medical care. It would be wonderful if we could measure a physician’s competence and quality and then reward the highest performers. Unfortunately, the concept of “quality” is a lot like the concept of “justice” – we know it when we see it, but no one can properly define it. Because “quality” can’t be measured, satisfaction survey companies instead take a variable that can be measured and use their “experts” to make everyone believe that this measurable variable is the most important aspect of medical care. In other words, they take a big pot of “patient satisfaction” and slap the label “quality” over the front of it – kind of like slapping the label “thermometer” on a ruler and using it to measure the temperature. Then, by showing hospital administrators and hospital boards how nearby hospitals are “performing better” on these statistically invalid metrics, patient satisfaction companies start “competition wars” and get a full scale buy-in from their clients to see who can be the “best.”

Satisfaction survey companies prey upon a hospitals’ desires to set themselves apart. This “Top 100” and that “Top 100” are plastered on billboards all over town. Satisfaction scores give hospitals yet another metric to brag about, but those scores ignore a patient’s quality of care. Medical judgment doesn’t matter as long as the numbers are at the 90th percentile.

Think about what satisfaction scores actually measure. Satisfaction scores don’t grade us on well we treat extremely sick patients. Admitted and transferred patients don’t get our surveys. Satisfaction scores generally don’t even measure how we treat a majority of our discharged patients. We all do a pretty darn good job at communicating with our patients and the numbers prove it. In a recent set of Press Ganey physician courtesy scores, doctors needed a mean score of 91.8 in order to be in the coveted 90th percentile. If their mean scores dropped only four points to 87.8, they found themselves in the loathsome 50th percentile. The grouping is tight which shows that most docs are behaving quite similarly.

What separates the “good” doctor from the “bad” doctor? Their ability to please difficult patients – the patients who have unreasonable expectations or who want inappropriate medical care. Give them what they want, or with a few pencil swipes, one angry patient can drag a physician from the 99th percentile to the tenth percentile. Think about it. Start with the scores of four patients who rated a physician with perfect “100s” and average in one patient with all “zero” ratings. The mean score falls from “100” to “80,” moving the physician from the 99th percentile to less than the 10th percentile on Press Ganey’s rankings. One patient can cause a change of 20 points, yet only 4 points separate the 50th from the 90th percentile? Houston, we have a problem.

Woe is the doctor who fails to admit a patient who wants to be admitted but who does not meet admission criteria. A coughing patient won’t accept your explanation why a Z-pack won’t help him? Turn him away at your own risk. With our employment and our compensation hinging on every “5” we can get, doctors are being coerced into giving patients whatever they want, regardless of medical appropriateness. When we cater to satisfaction scores more than we cater to proper medical care, we are violating our oath, devaluing our education, and potentially harming our patients.

Patient feedback is a tremendous asset in showing us how we can make our patients happier. But when we create large spreadsheets with green, yellow, and red percentile scores comparing statistically insignificant data about an unrelated set of criteria to manufacture some grand illusion that one hospital or one physician is “better” than another, we’re losing touch with reality.

Just ask my son.

William Sullivan is an emergency physician and an attorney and who blogs at Dr. William Sullivan’s Med Law Chronicles.

Submit a guest post and be heard on social media’s leading physician voice.

email

Comments are moderated before they are published. Please read the comment policy.

  • Stephen

    I work in the customer service department of a major teaching hospital. From my perspective there is great merit to what we have learned through HCAHPS and Press Ganey surveys.

    If the administration of your hospital is overly obtuse, then I can see why this would be a real problem. However, most hospitals have both quality metrics as well as patient satisfaction metrics. Even hospitalcompare.hhs.gov lets you see a segment of both; indicating that there is more to how good a hospital is than simply HCAHPS info, or Press Ganey scores. Good hospital administrators know you can’t sacrifice appropriate medical care for the sake of high patient satisfaction numbers. Sentinel events, lawsuits with merit, and JCAHO scrutiny, etc., would quickly overshadow any gains from sacrificing proper medical decisions for the sake of a better Press Ganey survey outcome.

    Also, every hospital deals with the “cough” patient and worse. Doing what is right will always earn every hospital a number of overly low survey scores. This is expected and understood by leadership with a proper knowledge of the workings of a hospital. That type of patient will give us a low rating whether we admit him or not. Why? Because at some point we are going to stop the medications that prompted him to come to the hospital in the first place. As soon as the medications stop the complaints begin. So, it is still better to make the proper medical decision since the low score is coming one way or the other.

    Usually, the next complaint about all of this I hear is that the patient population of a particular hospital is worse than all the other hospitals that have better scores. But, then again, I hear that from every unit’s nurses and physicians in the hospital. Everyone seems to have the worst patients and that always seems to be the reason for the scores they receive on their satisfaction surveys. Interestingly enough, the units who receive the highest scores spend a lot less time complaining at the nurses station and a whole lot more time providing appropriate patient care. The doctors who receive the highest scores provide great medical care and communicate effectively and efficiently with their patients and with other physicians and nurses.

    Is HCAHPS, Press Ganey, or any other survey system perfect? Hardly. I do believe that all survey companies owe an honest explanation of the numbers, including understanding of standard error given sample size. But I can wholeheartedly say that there has been a significant shift in patient interactions with both doctors and nurses in the three years we have been tracking HCAHPS data, and now administering Press Ganey surveys. Many doctors who used to have a notorious reputation for horrible bedside manner have dropped off our customer service radar – either by being removed or by changing. The same is true for many nurses who constantly caused complaints. Have our quality measured dropped? No! Good communication leads to less errors and less readmits. Should it require being held accountable to survey results (with monetary consequences) to get certain health care personnel to change their behavior? Of course not. But, the bottom line is that it has had that impact. And this doesn’t even take into account the impact it has had on improving environmental and dietary processes by focusing attention on those areas.

    In the end, the data collected on these surveys does matter. Questions about communication, timeliness, cleanliness and others are relevant to the patient experience and good medical care. These questions aren’t the full sum and can’t in and of themselves declare whether or not the care was appropriate, but neither can clinical outcomes alone be the measure of effectiveness and quality. Hopefully both care givers and hospital leadership can see the need for both and their place in healthcare.

  • http://myheartsisters.org Carolyn Thomas

    Very interesting topic – thanks so much for this.

    “Studies show a clear correlation between decreased patient satisfaction and increased medical malpractice risk.”

    Not only do such studies exist, but those who study doctor-patient communication for a living have a wealth of information about what specifically makes patients dissatisfied. Docs would do well to heed these lessons.

    For example, in a landmark 1997 study, the University of Toronto’s Dr. Wendy Levinson (considered among the world’s foremost researchers on physician-patient communication) recorded hundreds of conversations between a group of physicians and their patients. Half of the doctors had never been sued, and the other half had been sued at least twice.

    Levinson found that just on the basis of those recorded conversations alone, she could find clear differences between the two groups:

    * The doctors who had never been sued spent more than three minutes longer with each patient than those who had been sued did (18.3 minutes versus 15 minutes).

    * They were more likely to make “orienting” comments, such as “First I’ll examine you, and then we will talk the problem over” or “I will leave time for your questions.”

    * They were more likely to engage in active listening, saying things such as “Go on, tell me more about that.”

    Levinson reported NO DIFFERENCE in the amount or quality of clinical information doctors gave their patients; the never-sued doctors didn’t provide more details about medication or the patient’s condition.

    The difference was entirely in how they talked to their patients. U.K. malpractice studies out of St. Mary’s Hospital, London also found that patients are generally reluctant to sue doctors they like. For example, some reported intent to sue the specialist who referred them to the surgeon who botched the surgery – but not the surgeon himself (because “He’s a wonderful man!”)

    Interestingly, Dr. Wendy Levinson claims that, although her colleagues are often more focused on their medicine than their communication skills, she knows consumers can certainly relate to the need to enhance doctor-patient communication. She writes:

    “At cocktail parties, when I tell other doctors what I do, they’re not really interested. But if I tell patients, they ALL have a story to tell.”

    More on this at “Why Doctors Get Sued” on THE ETHICAL NAG: MARKETING ETHICS FOR THE EASILY SWAYED – http://ethicalnag.org/2010/10/15/why-doctors-get-sued/

  • http://www.drjshousecalls.blogspot.com Dr. Mary Johnson

    Timely post.

    Practicing in a smaller “sample” group, I have to question WHY a hospital staff that is REQUIRED BY LAW to report child abuse/child neglect on the part of parents (especially parents who are substance abusers) . . . and therefore might have less-than-fuzzy interactions in the process . . . can then be penalized by the bad feedback they get from said bad parents.

    And the bad feedback has the potential to lower our reimbursement – and therefore reduce our ability to provide quality care to people who aren’t crackheads/dope-dealers/child-abusers/etc. – people who appreciate what we do.

    No one has managed to explain it to me yet.

  • Marc Gorayeb, MD

    Compare the cogency and logic of the author’s post with that of respondent S.
    Dr. Sullivan exposes the invalidity of the survey methodology. He succinctly demonstrates that the survey questions fail to measure true quality, that the score can be terribly skewed by just a few outliers, and that administrators use the results improperly.

    Respondent S., on the other hand, avoids the central theme of the argument.
    First, he claims that most hospitals have both quality metrics and patient satisfaction metrics, and that “good” administrators keep the results in proper perspective. What quality metrics? How have they been validated? And how many “good” and wise administrators have you met in your travels?
    S. refers to patient satisfaction “metrics.” Dr. Sullivan just proved that these are not “metrics” at all. S. fails to address this most critical issue.
    S. claims that the complaining patient will inevitably complain at some point along the line; so why not get it over with and take one for the team, ER doctor? What planet is this again?
    S. makes the assertion that high survey score recipients spend more time caring for patients, spend less time complaining at the nursing station, provide great medical care and communicate effectively and efficiently with their patients and with other physicians and nurses. Wow! It must be true because he said so. Who needs analysis like Dr. Sullivan’s, when S.’s feeelings give us the answer?
    S. says that good communication leads to less errors and less readmits. A generic platitude thrown in to support his argument without showing how it applies. What kind of communication specifically perceived by the patient leads to fewer errors and readmits? Where are the data and the analysis to connect this to patient satisfaction surveys?
    S. says that questions about communication, timeliness, cleanliness and others are relevant to the patient experience and good medical care. Notice how he appended the phrase “…and good medical care” to the end of a self-evident statement? Nice technique of proving a conclusion by simply stating the conclusion. (..and confusing the “quality metrics” with “satisfaction metrics” that he so carefully distinguished at the beginning of his response).
    So there you have it: an analytical and persuasive piece answered by inapposite and meandering counterarguments that nevertheless feeel right.

    And as for the argument that patients are less likely to sue doctors who communicate well with them; that may help the doctor and hospital to stay out of trouble, but it doesn’t guarantee that the patient will stay out of trouble. Just remember that physicians with mediocre intellect and skills are able to survive by being great ‘communicators.’ Be cautious using that characteristic as a significant factor in choosing your doctor (or hospital).

    • http://myheartsisters.org Carolyn Thomas

      Re medmal suits: “….Just remember that physicians with mediocre intellect and skills are able to survive by being great ‘communicators’….”

      And as the research shows, even physicians who may be brilliant clinicians get sued more if they are lousy communicators.

      You pick.

  • http://glasshospital.com GlassHospital

    Dear Stephen:

    unlike Dr. G, I think your comments are on the mark. I’d love to chat with you further. Please contact me if you’re willing.

    Many thanks-
    John Schumann MD
    University of Chicago

  • Pauline Wong-Felix

    I fully agree with you Stephen, I am a nurse and I also work in Customer Service in a hospital. One can learn alot from those HCAHPS scores. How the patient feels and complaints should not be taken negatively. We should learn from complaints and wisely recover from negative responses. We should also question ourselves, if we are on the receiving end of this service at the hospital how we feel; I pose this question to nurses and doctors, would you choose the nurseor doctor who is like you? If we spend time to listen to our patients, give them the kind of education they should have and keep them informed, I think HCAHPS should pose no problem at all. Times have changed and patients want to be informed and make decisions; they need to be fully informed and be active paticipants in their own treatment. When we brush aside customer service and think that we can still go on as we have been before, we are in for a rude awakening, as patient satisfaction does pose a problem with reimbursement. This is an article I wrote in the Advance Nursing regarding how as nurses we can do out bit in providing quality customer service to our patients.
    Providing Customer Service … Easily
    By Pauline Wong-Felix, LVN
    Emerging trends in healthcare have made great impacts on the quality of nursing – especially when it comes to customer service.

    Proper customer experiences are very important in today’s trend of insurance reimbursement of services. Hospital Consumer Assessment of Healthcare Providers & Systems (HCAHPS) tracks hospital performance and can be viewed online by the general public.

    As nurses, we need to ask ourselves what we can do to make a difference to contribute towards quality care in our profession.

    Self-Evaluation
    What do you do when you get a shift-change report that leads you to believe you’ll be caring for a very difficult – maybe a patient who’s hard to please?

    Do you give up on the patient? What goes through your mind? Are you defensive, discriminatory or judgmental with unspoken feelings of resignation, or do you find ways to get through to these individuals? What is your reaction when the patient asks for help?

    We tend to ignore or avoid these patients to prevent conflicts because these “trouble makers” deter us from completing our jobs. We might also feel resigned to just administer their medications, run out of the room so there will be no issues, finish our jobs and go home.

    But we should be questioning ourselves as to why it is we’re getting such negative response from our patients. We should be trying to find the root of the problem that is bugging our patients instead of branding them as “difficult.”

    When we are sick and uncertain of what is happening to our own health, what image of a nurse enters through our minds when we think about the care we need? Would you choose a nurse who is like you?

    We deal and face different difficulties in life, including those in the work force. How we deal with these difficulties when we are faced with them defines the customer service we provide.

    Focus on the Positive
    Stop to think about the last time a patient said “thank you.” If you can hold onto that good feeling and work to do the best you can to get that compliment again, odds are you will be performing positive customer service. But expecting a patient to tell you how great a nurse you are by robotically giving them medications and walking away without so much as a word or smile will not jive.

    Beneficial customer service also means allowing the patient to understand that you do have the time for them, specifically. They can tell that by your body language. We are all from the same consciousness, we feel one another whether you know it or not.

    A smile, a light touch, holding a patient’s hand and telling him you are there to help can go a long way in bonding with the patient and making your job more enjoyable.

    An easy way to ensure that the customer service you provide is what’s expected is to always question yourself as to why you became a nurse in the first place. Is it because society thinks nursing is a well-respected profession, or is it out of the compassion you have for the sick and suffering?

    Whenever you feel harassed or not in sync, think of how you can make a difference in someone’s life when they’re sick and dependant on you.

    Listen to your heart and recall any negative response that you get from your patients. List those responses and critically think about how you can do better, even if you believe the patient is being difficult.