Finding a doctor using ratings is a sound idea, but poorly executed

Let’s say you develop some heart problem and get sent to me for an evaluation. You show up to the office, check in, get ushered into an exam room, then you wait and wait and wait. When I finally come into the room I seem terse, impatient and rushed, and you end up visiting with me for less than 5 minutes. I provide you with a diagnosis, order a couple of tests and dash off a prescription.

What’s your impression of me? What’s your opinion of me as a doctor, or of my medical skills? If you were asked to rate the quality of care you receive what kind of score would you give me?

Now let’s add some more information to the mix. Would it change your mind to learn that I was late seeing you because I was in the emergency department providing lifesaving care to Sister Margaret Mary, an elderly nun who is the sole caregiver for a dozen homeless orphans? And I cut my visit short because I had to get back to the cath lab to open her critically blocked heart artery so that she can survive to continue her motherly care and what seemed like impatience was actually my anxiety about the welfare of the poor orphaned children.

Would your opinion of me be any different?

And what if you see me for a follow-up visit on a day when I am not busy saving nuns or orphans, have all the time in the world for you, and yet I indifferently rush through my appointment just so I can get back to my office to surf the internet and update my Facebook profile? Are you now back to your original opinion of me?

My real question for you is this: Given the limited interaction you’ve experienced, are you in a good position to pass judgment on my skills as a doctor?

According to many websites on the Internet, you are.

Go to your favorite search engine and type in “rate your doctor” and you’ll find a dozen sites that allow you to grade your doctor on knowledge, timeliness, professionalism, competency, and even compassion. The rise of these sites mirrors other service industries—hotels, restaurants, home repairpersons, etc.—that have had to begin to factor open, anonymous, internet-based feedback into their management strategies. One area where this has become amazingly popular is education, where students can now seek revenge on boring instructors who were not insightful enough to reward the student’s languid brilliance with anything more than a C-.

In order to come to some understanding of this trend I spent a little time browsing my own entry on various doctor rating websites and have come to the conclusion that they tend to be inaccurate, dated, and sparsely applied.

I learned from RateMDs.com that my name is actually Dr. Van and that I practice in Lincoln (my office used to be headquartered there). No one has provided any feedback on Dr. Van, although one person gave a glowing report to my brother, Dr. Van De Graaff, a dermatologist in Omaha (whom I easily found): “Staff was super nice as was he, and he didn’t waste my time. He isn’t out for the money as he even recommended the least expensive option which worked great. I would totally recommend him.” This anonymous rater, who thought my big brother was “super nice,” obviously didn’t grow up with him.

Vitals.com also believes I practice in Lincoln but at least they got my last name right. In addition to the usual demographic information they provide about me they also go to the trouble to rate my medical school (3 stars) and my residency training program (4 stars).

DrScore doesn’t even know who I am, but did identify another sibling of mine as a head and neck surgeon in Lincoln (unfortunately, he moved to Idaho over two years ago). He hasn’t had anyone rate him yet, although I was tempted to grade him on his competence as a little brother.

DoctorScorecard and Healthcare Reviews advertise themselves as rating sites but are pretty shoddy. To be fair, though, they seemed to target an international audience by providing ratings for physicians in India and the U.K. Needless to say I couldn’t find my name on these sites.

The very popular Angie’s List has come to doctor rating a little late in the game. This is the only site that requires you to pay a subscription to search for your doctor (the other sites will give you basic information for free but you’ll have to pay a small fee to learn if your doctor has malpractice suits or is wanted by the police). I can’t give you any feedback about Angie’s List since I’m too much of a cheapskate to sign up for service.

The most popular site for doctor rating appears to be HealthGrades (another site, PhysicianReports, links straight into HealthGrades). I was able to find my name along with current addresses for our Alegent offices, although I had to search under “Lincoln” to locate myself. Miraculously I learned that one patient went to the trouble to locate me on this site and provide a rating (I got 5 stars in all categories, which makes me suspicious of my 8-year-old daughter and her precocious internet skills).

Thinking that perhaps my patients are less internet-savvy (or perhaps more indifferent to the care I provide) I browsed HealthGrades to see if other doctors in Omaha have more patient feedback that I have. Of the 84 cardiologists listed, only 36 showed any rating at all, with most of those having only a solitary patient who went to the trouble to find and grade them. Only one cardiologist had ratings from 3 patients.

Our experience in Omaha doesn’t differ significantly from the trend nationwide.

A study in last month’s Journal of General Internal Medicine showed that Boston-area practitioners were as rarely reviewed as my colleagues and I are. Additionally, almost all the reviews given where overwhelmingly glowing—not much help if you want to weed out the doctors you want to avoid. A commentary in the LA Times summed up the overall impact of these sites:

Patients just don’t appear too interested in providing feedback on their doctors, the authors noted, despite the fact that consumers generally love and use ratings systems. In contrast to the scarcity of doctor reviews, a search of restaurants in Boston’s Beacon Hill area “turned up 38 narrative reviews for a single Lebanese restaurant,” the authors pointed out.

The sites have the potential to empower patients. And, to be sure, consumers today are encouraged to be smarter and more discriminating shoppers of healthcare services and products in order to reduce costs. But this is one Internet function that doesn’t seem to do patients much good or doctors much harm.

From a statistical perspective it makes sense that these ratings sites have yet to achieve prime-time utility. Just one or two reviews on a doctor are not enough to be able to provide a representative opinion on the overall care of the provider. A single, heavily biased review (whether good or bad) would impart an inordinate influence on the overall score. In order for such a system to really provide useful feedback it would have to include reviews from hundreds of patients (such as is the case with consumer websites such as Amazon), a scenario we’re not likely to see anytime soon.

My verdict is that the idea as a whole is sound but that the execution thus far is lacking. I feel comfortable that my value as a service provider could be accurately scored and quantified if it were based on the feedback of hundreds of my patients—if I’m a lousy doctor the truth will undoubtedly come out in the overall tally. Just one perfect score (like I currently have on HealthGrades) doesn’t give my prospective patients any useful information about what they might expect when they come to see me. Someday this system might be an excellent way for patients to have more control over what sort of provider interaction they can expect, but as it stands they still have to rely on old-fashioned word-of mouth and hope for the best.

Now, if you’ll excuse me, I’ve got to get back to caring for nuns and orphans.

Eric Van De Graaff is a cardiologist at Alegent Health who blogs at the Alegent Health Cardiology Blog.

Submit a guest post and be heard.

Comments are moderated before they are published. Please read the comment policy.

  • r watkins

    Any patient who chooses their physician based on anonymous comments on an unmonitored website is a fool.

  • HJ

    So my life doesn’t have the same value as a nun or orphan?

    • PCP MD

      Not necessarily. Some lives are indeed more valuable to society than others. Take the President of the United States, at any given time, for example.

      • http://www.twitter.com/alicearobertson Alice

        The current president or previous ones? :)

  • http://www.angieslist.com Cheryl from Angie’s List

    Great post, Dr. De Graaff. I’m Cheryl Reed from Angie’s List, and I wanted to say that you make some excellent points.

    Because you didn’t have direct information from Angie’s List, I thought I’d point out that if a patient were to have submitted a report on Angie’s List using your scenario, you would have the opportunity (free of charge by the way) to respond to the patient either directly or by posting a response to the report so other members would get both sides of the story.

    We don’t allow anonymous reports and we hold members accountable for truthful reporting.

    We encourage service professionals to respond to all reports — positive or negative — and to actively engage in the Angie’s List process.

    We agree that it would be great if there were dozens of reports about every doctor and service professional, and we’re working toward that goal. In the meantime, we believe that even a little information is better than none.

    Good luck with the nuns, the orphans and your siblings, as well!

    • Eric Van De Graaff

      Cheryl,

      Thanks for posting a response to my article. I hope you don’t take offense at my tongue-in-cheek ribbing of the doctor-rating community. I agree with you that doctors should be routinely rated and I hope services like yours will someday provide a clearinghouse for accurate information that can help steer patients towards doctors who can provide the best care. As with nearly everything, transparency can only improve the service in the world of health care.

      And, you’re right: it’s my siblings I struggle with most.

      Dr. VDG

  • http://www.pacificpsych.com/ pacificpsych

    Do you mean, as an example…

    Cheryl (as a patient): Pacificpsych was horrible and awful and mean. She spent one second with me!

    Pacificpsych: That’s not true! Cheryl is borderline and obnoxious. I spent 2 hours with her–and she didn’t even pay!

    Yup, this should work well…

  • http://retiredbicycle.blogspot.com Allison

    Do you ever read comments on Yahoo articles? Those are the same people who are writing reviews of just about everything. So, no, they would not influence my choice of a doctor. However, if you’re running late because you’re saving the nun, have someone in the office TELL the patient why you’re late, and how late you’re going to be. Patient can wait or reschedule, but at least patient knows why you aren’t there. When you do show up, apologize, tell patient you’re going to run tests for more data and will see patient again armed with diagnostics. Patients are generally smart enough to know that OB/GYNs and cardiologists who perform surgery are sometimes called away.

  • http://Www.Twitter.com/alicearobertson Alice

    I really enjoyed this article and the facetious ending about the “nuns” and “orphans”.   Quite good….point taken…….but anonymity with posting is quite common among doctors….might want to go looking in your own backyard when you play hide-and-seek.   :)

  • http://www.angieslist.com Cheryl at Angie’s List

    No offense taken at all. You make good points and we want more information, too! I liked your post a lot — for a lot of reasons. I’m the youngest of seven; I can relate!

  • http://www.pacificpsych.com/ pacificpsych

    Pity my comment was not adresses. Let me rephrase the point.

    1) Some patients are not nice. That’s the nature of the business. They see bad intentions where there are nothing but good ones. That’s the *reason* some people end up going to psychiatrists. If they were to express their anger on rating sites on the Net, what do you think will happen? Why would anyone take on such a patient, only to have their reputation tarnished?

    2) Physicians CAN’T respond to patients. Psychiatrists CAN’T reveal who their patients are. No physician can. It would be extremely unprofessional, not to mention grounds for losing your license and getting sued, to respond personally to a personal attack by a patient on a public site.

    Physicians and patients are not on equal footing. Patients can say whatever they want, to whomever they want. Physicians can’t.

  • http://www.twitter.com/alicearobertson Alice

    Physicians and patients are not on equal footing. Patients can say whatever they want, to whomever they want. Physicians can’t.
    [end quote]

    As a patient I would agree with this, but pity on either side of the fence isn’t helpful. It is not a level playing field as long as doctors accept money, and their client is a type of customer/patient seeking counsel. It’s a whole different playing field these days…….thankfully, we aren’t in Kansas anymore and doctors are acclimating and thrashing about just as patients used to (and often still do……there are different flavors of doctors and some may be victims, and some create victims).

  • http://www.pacificpsych.com/ pacificpsych

    Yes, we’re not in Kansas anymore. That doesn’t mean what you think, though. It means that white coats, managed care, big pharma and bureaucrats have won. It’s sad.

    “It is hard for a free fish to understand what is happening to a hooked one”. – Karl Menninger, M.D., Topeka, KS.

  • http://www.twitter.com/alicearobertson Alice

    mment Yes, we’re not in Kansas anymore. That doesn’t mean what you think, though. It means that white coats, managed care, big pharma and bureaucrats have won. It’s sad”It is hard for a free fish to understand what is happening to a hooked one”. – Karl Menninger, M.D., Topeka, KS [end quote]

    Hmmm…..are you a vegetarian? :) Okay…..after my moment of silliness….I get it……bruised doctors….but I do understand…….and agree the peripherals of medicine are a pain…..yet, with a daughter with cancer I am not objective when I am so grateful for the way mankind has gained from the problematic free market, etc., but you gotta admit some of the regulation (that drives me crazy as well…because of my libertarian leanings) came about because of doctors beahaving badly.

  • Tamara Greet

    Found this very interesting. I have been arguing back and forth with Yelp regarding this very issue. If a patient comes in and let’s say (completely hypothetically speaking) is seeking drugs and we don’t give them what they want, they can go to a website and leave a negative comment. Because of HIPPA, I can’t add my own comment regarding the patient’s experience, which is frustrating.

    The tried and true method of finding any good physician is word of mouth….but not from anonymous sources.

  • http://www.angieslist.com Cheryl from Angie’s List

    We realize there are some limitations on how physicians can respond, Dr. Greet, and we worked hard to address those. Angie’s List does a few things on this front, and we’re always open to more discussion and ideas:
    1. Our reports aren’t anonymous, so we know who’s submitting reports, and if the physician or other kind of service provider asks us, we’ll put them in touch with the member so they can directly communicate about the report. (We remind members of this potential action every time they submit a report.)
    2. We have a process in place to allow commentary that doesn’t violate HIPAA .
    3. We believe there are ways for physicians to respond to reports without violating HIPAA, and we’re happy to help any service provider to post a response that will give our members both sides of any story.

  • Tamara Greet

    I am not a doctor. I’m an office administrator. I run an office for six physicians.

  • gzuckier

    My own experience matches that of the author; attempting to find criteria on which to choose a specialist from a few local possibilities, the rating sites on the Web offer no mention for most, and the few reviews that did appear tended very strongly to fall into the category of “crazed rant” which has become a preferred means of Web-based communication.

  • http://www.pacificpsych.com/ pacificpsych

    There is no way on earth that a psychiatrist can respond to an online pt complaint. No way. You’d be violating your patient’s privacy, which is absolute save for certain law enforcement and mental health situations. You’d be setting yourself up to lose your license. And you’d make yourself look like a fool.

    “we’ll put them in touch with the member so they can directly communicate about the report. ”

    This is just so wrong, in so many ways…

  • doctor

    Good post. Most patients still choose by word of mouth from a trusted friend, which has more value. Certain doctors are better fits with different patients, and you can’t put a numerical rating on this. Furthermore, these sites are open to all kinds of manipulation, as the author notes, and are overly influenced by those most inclined to fill out a rating- sometimes those who are very happy or very angry, but mostly those with a lot of time on their hands.

Most Popular