Skip to content
  • About
  • Contact
  • Contribute
  • Book
  • Careers
  • Podcast
  • Recommended
  • Speaking
  • All
  • Physician
  • Practice
  • Policy
  • Finance
  • Conditions
  • .edu
  • Patient
  • Meds
  • Tech
  • Social
  • Video
    • All
    • Physician
    • Practice
    • Policy
    • Finance
    • Conditions
    • .edu
    • Patient
    • Meds
    • Tech
    • Social
    • Video
    • About
    • Contact
    • Contribute
    • Book
    • Careers
    • Podcast
    • Recommended
    • Speaking

Why ProPublica’s Surgeon Scorecard is worth celebrating

Ashish Jha, MD, MPH
Physician
July 26, 2015
257 Shares
Share
Tweet
Share

shutterstock_153912641

Recently, Marshall Allan and Olga Pierce, two journalists at ProPublica, published a surgeon report card detailing complication rates of 17,000 individual surgeons from across the nation. A product of many years of work, it benefitted from the input of a large number of experts (as well as folks like me). The report card has received a lot of attention … and a lot of criticism. Why the attention? Because people want information about how to pick a good surgeon. Why the criticism?  Because the report card has plenty of limitations.

As soon as the report was out, so were the scalpels. Smart people on Twitter and blogs took the ProPublica team to task for all sorts of reasonable and even necessary concerns. For example, it only covered Medicare beneficiaries, which means that for many surgeries, it missed a large chunk of patients. Worse, it failed to examine many surgeries altogether. But there was more.

The report card used readmissions as a marker of complications, which has important limitations. The best data suggest that while a large proportion of surgical readmissions are due to a complication, readmissions are also affected by other factors, such as how sick the patient was prior to surgery (the ProPublica team tried to account for this), his or her race, ethnicity, social supports — and even the education and poverty level of their community. I have written extensively about the problems of using readmissions after medical conditions as a quality measure. Surgical readmissions are clearly better but hardly perfect. They even narrowed the causes of readmissions using expert input to improve the measure, but even so, it’s hardly ideal. ProPublica produced an imperfect report. 

How to choose a surgeon

So what to do if you need a surgeon?  Should you use the ProPublica report card?  You might consider doing what I did when I needed a surgeon after a shoulder injury two years ago: ask colleagues. After getting input about lots of clinicians, I honed in on two orthopedists who specialized in shoulders. I then called surgeons who had operated with these guys and got their opinions. Both were good, I was told, but one was better. Yelp?  I passed. Looking them up on the Massachusetts Registry of Medicine?  Seriously?  Never crossed my mind.

But what if, just by chance, you are not a physician? What if you are one of the 99.7 percent of Americans who didn’t go to medical school? What do you do?  If your insurance covers a broad network and your primary care physician is diligent and knows a large number of surgeons, you may get referred to someone right for you. Or, you could rely on word of mouth, which means relying on a sample size of one or two.

So what do patients actually do?  They cross their fingers, pray, and hope that the system will take care of them. How good is that system at taking care of them? It turns out, not as good as it should be. We know that mortality rates vary three-fold across hospitals. Even within the same hospital, some surgeons are terrific, while others? Not so much. Which is why I needed to work hard to find the right orthopedist. Physicians can figure out how to navigate the system. But what about everyone else?

I was on service recently and took care of a guy, Bobby Johnson (name changed, but a real guy), who was admitted yet again for an ongoing complication from his lung surgery. He had missed key events because of his complications — including his daughter’s wedding — because he was in the hospital with a recurrent infection. He wondered if he would have done better with a different hospital or a different surgeon. I didn’t know how to advise him.

And that’s where ProPublica comes into play. The journalists spent years on their effort, getting input from methodologists, surgeons, and policy experts. In the end, they produced a report with a lot of strengths, but no shortage of weaknesses. But despite the weaknesses, I never heard them question whether the endeavor was worth it at all.  I’m glad they never did.

Because the choice wasn’t between building the perfect report card and building the one they did. The choice was between building their imperfect report card and leaving folks like Bobby with nothing. In that light, the report card looks pretty good. Maybe not for experts, but for Bobby.

A step towards intended consequences

 Colleagues and friends that I admire, including the brilliant Lisa Rosenbaum, have written about the unintended consequences of report cards. And they are right. All report cards have unintended consequences. This report card will have unintended consequences. It might even make, in the words of a recent blog, “some Morbidity Hunters become Cherry Pickers” (a smart, witty, but quite critical piece on the ProPublica Report Card). But asking whether this report card will have unintended consequences isn’t the right question. The right question is — will it leave Bobby better off? I think it will. Instead of choosing based on a sample size of one (his buddy who also had lung surgery), he might choose based on sample size of 40 or 60 or 80. Not perfect. Large confidence intervals? Sure. Lots of noise?  Yup. Inadequate risk adjustment? Absolutely. But, better than nothing? Yes. A lot better.

All of this gets at a bigger point raised by Paul Levy:  Is this really the best we can do? The answer, of course, is no. We can do much better, but we have chosen not to. We have this tool — it’s called the National Surgical Quality Improvement Program (NSQIP). It uses clinical data to carefully track complications across a large range of surgeries, and it’s been around for about twenty years. Close to 600 hospitals use it (and about 3,000 hospitals choose not to). And no hospital that I’m aware of makes its NSQIP data publicly available in a way that is accessible and usable to patients. A few put summary data on Hospital Compare, but it’s inadequate for choosing a good surgeon. Why are the NSQIP data not collected routinely and made widely available? Because it’s hard to get hospitals to agree to mandatory data collection and public reporting. Obviously those with the power of the purse — Medicare, for instance — could make it happen. They haven’t.

Disruptive innovation, a phrase coined by Clay Christensen, is usually a new product that, to experts, looks inadequate. Because it is. These innovations are not, initially, as good as what the experts use (in this case, their network of surgeons). They initially dismiss the disrupter as being of poor quality. But disruptive innovation takes hold because, for a large chunk of consumers (i.e., patients looking for surgeons), the innovation is both affordable and better than the alternative. And once it takes hold, it starts to get better. And as it does, its unintended consequences will become dwarfed by its intended consequences:  making the system better. That’s what ProPublica has produced.  And that’s worth celebrating.

Ashish Jha is an associate professor of health policy and management, Harvard School of Public Health, Boston, MA.  He blogs at An Ounce of Evidence and can be found on Twitter @ashishkjha.

Image credit: Shutterstock.com

Prev

Analyzing the deficit of African-Americans in academic medicine

July 25, 2015 Kevin 40
…
Next

I will not let a computer come between me and my patient

July 26, 2015 Kevin 23
…

Tagged as: Surgery

Post navigation

< Previous Post
Analyzing the deficit of African-Americans in academic medicine
Next Post >
I will not let a computer come between me and my patient

More by Ashish Jha, MD, MPH

  • Ranking the world’s health systems: These results may surprise you

    Ashish Jha, MD, MPH
  • How much does it matter which hospital you go to?

    Ashish Jha, MD, MPH
  • Men and women doctors versus correlation and causation

    Ashish Jha, MD, MPH

Related Posts

  • A physician’s addiction to social media

    Amanda Xi, MD
  • Why creative endeavors are important for the future surgeon

    Thomas L. Amburn
  • Medicine was consuming this medical student. Was it worth it?

    Sarah B. El Iskandarani
  • Paging the surgeon general: America needs you

    Linda Girgis, MD
  • The irony of me observing death on a day celebrating my birth

    Johnathan Yao, MD, MPH
  • A trauma surgeon reflects on the Yale System, 20 years later

    Ara Feinstein, MD, MPH

More in Physician

  • Physicians have no autonomy. Here’s how to change that.

    Diane W. Shannon, MD, MPH
  • The erosion of patient care

    Laura de la Torre, MD
  • Navigating adulthood in the digital age

    Eleanor Menzin, MD
  • The power of business knowledge for medical professionals

    Curtis G. Graham, MD
  • Using the language of art to create work-life balance

    Sarah Samaan, MD
  • Lively communication in the service industry

    Deepak Gupta, MD
  • Most Popular

  • Past Week

    • Reigniting after burnout: 3 physician stories

      Kim Downey, PT | Physician
    • The erosion of patient care

      Laura de la Torre, MD | Physician
    • I’m tired of being a distracted doctor

      Shiv Rao, MD | Tech
    • Inside the grueling life of a surgery intern

      Randall S. Fong, MD | Physician
    • How Tratak yoga reshaped my USMLE Step 2 prep

      Dr. Nikita Mehdiratta | Education
    • Family support is pivotal in the treatment of schizophrenia

      Frank Chen, MD | Conditions
  • Past 6 Months

    • Medical gaslighting: a growing challenge in today’s medical landscape

      Tami Burdick | Conditions
    • I want to be a doctor who can provide care for women: What states must I rule out for my medical education?

      Nandini Erodula | Education
    • Balancing opioid medication in chronic pain

      L. Joseph Parker, MD | Conditions
    • Reigniting after burnout: 3 physician stories

      Kim Downey, PT | Physician
    • I’m a doctor, and I almost died during childbirth

      Bayo Curry-Winchell, MD | Physician
    • Mourning the silent epidemic: the physician suicide crisis and suggestions for change

      Amna Shabbir, MD | Physician
  • Recent Posts

    • How Tratak yoga reshaped my USMLE Step 2 prep

      Dr. Nikita Mehdiratta | Education
    • Transforming primary care for physician well-being [PODCAST]

      The Podcast by KevinMD | Podcast
    • COVID-19 unleashed an ongoing crisis of delirium in hospitals

      Christina Reppas-Rindlisbacher, MD, Nathan Stall, MD, and Paula Rochon, MD | Conditions
    • Doctors and disability insurance: Protecting your income

      Amarish Dave, DO | Finance
    • Emergency care nightmare: the urgent need for experienced nurses

      Rachel Basham, RN, CCRN | Conditions
    • Physicians have no autonomy. Here’s how to change that.

      Diane W. Shannon, MD, MPH | Physician

Subscribe to KevinMD and never miss a story!

Get free updates delivered free to your inbox.


Find jobs at
Careers by KevinMD.com

Search thousands of physician, PA, NP, and CRNA jobs now.

Learn more

View 12 Comments >

Founded in 2004 by Kevin Pho, MD, KevinMD.com is the web’s leading platform where physicians, advanced practitioners, nurses, medical students, and patients share their insight and tell their stories.

Social

  • Like on Facebook
  • Follow on Twitter
  • Connect on Linkedin
  • Subscribe on Youtube
  • Instagram

CME Spotlights

From MedPage Today

Latest News

  • Report: Persistence of Gender Inequalities in Cancer Care, and a Call to Action
  • Cancer Risk in NAFLD Higher With Early Disease Onset
  • FDA Displeased With Companies Purposely Adding Sesame to More Foods
  • COVID Vax Appointment Cancelled? New Shot Rollout Faces Challenges
  • Medical Residents Receive 100+ Job Offer Contacts, Survey Shows

Meeting Coverage

  • New Schizophrenia Treatments Are Coming: Don't Panic
  • Loneliness Needs to Be Treated Like Any Other Health Condition, Researcher Suggests
  • Stopping Medical Misinformation Requires Early Detection
  • AI Has an Image Problem in Healthcare, Expert Says
  • Want Better Health Outcomes? Check Out What Other Countries Do
  • Most Popular

  • Past Week

    • Reigniting after burnout: 3 physician stories

      Kim Downey, PT | Physician
    • The erosion of patient care

      Laura de la Torre, MD | Physician
    • I’m tired of being a distracted doctor

      Shiv Rao, MD | Tech
    • Inside the grueling life of a surgery intern

      Randall S. Fong, MD | Physician
    • How Tratak yoga reshaped my USMLE Step 2 prep

      Dr. Nikita Mehdiratta | Education
    • Family support is pivotal in the treatment of schizophrenia

      Frank Chen, MD | Conditions
  • Past 6 Months

    • Medical gaslighting: a growing challenge in today’s medical landscape

      Tami Burdick | Conditions
    • I want to be a doctor who can provide care for women: What states must I rule out for my medical education?

      Nandini Erodula | Education
    • Balancing opioid medication in chronic pain

      L. Joseph Parker, MD | Conditions
    • Reigniting after burnout: 3 physician stories

      Kim Downey, PT | Physician
    • I’m a doctor, and I almost died during childbirth

      Bayo Curry-Winchell, MD | Physician
    • Mourning the silent epidemic: the physician suicide crisis and suggestions for change

      Amna Shabbir, MD | Physician
  • Recent Posts

    • How Tratak yoga reshaped my USMLE Step 2 prep

      Dr. Nikita Mehdiratta | Education
    • Transforming primary care for physician well-being [PODCAST]

      The Podcast by KevinMD | Podcast
    • COVID-19 unleashed an ongoing crisis of delirium in hospitals

      Christina Reppas-Rindlisbacher, MD, Nathan Stall, MD, and Paula Rochon, MD | Conditions
    • Doctors and disability insurance: Protecting your income

      Amarish Dave, DO | Finance
    • Emergency care nightmare: the urgent need for experienced nurses

      Rachel Basham, RN, CCRN | Conditions
    • Physicians have no autonomy. Here’s how to change that.

      Diane W. Shannon, MD, MPH | Physician

MedPage Today Professional

An Everyday Health Property Medpage Today
  • Terms of Use | Disclaimer
  • Privacy Policy
  • DMCA Policy
All Content © KevinMD, LLC
Site by Outthink Group

Why ProPublica’s Surgeon Scorecard is worth celebrating
12 comments

Comments are moderated before they are published. Please read the comment policy.

Loading Comments...