The shocking lack of data behind the medical home

When looking to examine patient-centered medical home data, there’s no better place to start than the 26-month evaluation of the multi-site National Demonstration Project. Begun in 2006, with results published in 2010, this federally funded project included an array of repeated cross-sectional surveys and medical record audits at baseline, 9 months, and 26 months, using patients from 36 family practices that were randomized into two groups: those practices that received facilitation to become PCMHs and those self-directed in their PCMH adoption.

After all this money and time, the study found “no significant differences between groups” and “no improvements in patient-rated outcomes.” In fact, the only change found was minor improvement in scores on surveys that providers did about their care. Or, as the conclusion notes, “After slightly more than 2 years, implementation of PCMH components, whether by facilitation or practice self-direction, was associated with small improvements in condition-specific quality of care but not patient experience.”

New York state’s Medicaid program, which includes over 5 million enrollees and over 13,000 primary care physicians, markedly expanded payment for PCMHs, beginning in 2010. The summary report from April 2013 is, in many ways, both typical and chilling.

After a glowing introduction and overview, including a background section that emphasizes the potential of PCMHs (without discussing the evidence, or lack thereof), the document has some notable findings. Three years into the program, despite an impressive amount of incentive money, only 34% of primary care providers were willing to become PCMHs. The graph on page 9 shows the growth in participation flattening out after one year. By mid-2012, 38% of New York state’s 5 million Medicaid patients were part of the project through their doctor’s participation.

Between 2010 and 2012, New York spent an estimated $398,947,964 taxpayer dollars (which includes a $250 million lump sum payment to hospitals and training centers) — money designated for the care of New York’s poorest, and often most medically fragile and disabled patients — to implement this certification bureaucracy.

But for all the money and effort spent on New York’s PCMH bureaucracy, starting on page 14 of the report are some very underwhelming results. Most striking is the difference between the generally glowing written summary in favor of PCMHs (e.g., “These analyses show that PCMH practices have higher rates of quality performance, as defined by national standardized measures, than non-PCMH practices for a majority of measures after controlling for differences in enrollee case mix.”), and the actual reported numbers.

Minor differences are noted, with simple adult and pediatric BMI measurements being the rare and largest difference in the groups to favor PCMHs. In many cases, non-PCMH providers performed equally as well as PCMH ones in preventive services, and outperformed them with a large difference in non-PCMH providers’ “avoidance of antibiotics therapy in adults with acute bronchitis.”

Conspicuously, data show that patients in PCMHs had higher rates of ED visits as well as higher rates of both overall and preventable admissions to the hospital. Here’s how the shocking results are described in the discussion: “The utilization results, however, while preliminary, do not at this time show changes in the expected, or desired reductions in ER visits or inpatient stays.” It’s unclear why utilization results are deemed preliminary, but the outcome data are not. Nor is it clear why “at this time” is a pertinent modifier for only these poor results. Patient satisfaction results, a core component of evaluating patient-centeredness, is also notably missing, explained away by “as surveys are based on a sample of enrollees, 65% of whom do not respond, there is often not enough data to draw meaningful conclusions.”

In terms of other large PCMH projects, the Safety Net Medical Home Initiative was one of the largest nationwide PCMH implementation programs. Evaluation of it showed that staff resistance and turnover were obstacles. A more in-depth look at 5 safety net clinics in New Orleans showed more pressing problems than PCMH status, including “a need to focus on clinic finances.”

One notable study shows a marked benefit from PCMH implementation in a military center, with 7% improvement in access to care, a whopping 75% decrease in ED utilization and increased staff satisfaction scores after two years. Despite this, a larger VA study found more challenges and variability in just getting programs implemented, particularly around issues of open access to care. These conflicting results at clinics that operate within a military culture point to the difficulty in creating a standardized successful PCMH implementation.

An overview of published PCMH studies generally finds the data tending toward positive results, but plagued by “methodological and measurement issues,” a sentiment echoed by another review of the evidence. Drilling down, a large study of 58,391 patients seen at one of 22 medical groups between 2005-2009 found that any cost savings were limited to only the most medically complex, with some net increases in utilization and costs among other groups. A study of 27 Minnesota-based medical groups found little to no overall correlation between PCMH and diabetes care costs. Two Group Health Cooperative reports, from a study that is frequently touted as demonstrating “the potential” of PCMHs, when viewed critically, actually showed spotty improvements in some patient experience scores, no significant differences in outcomes, and no difference in overall costs with PCMHs, even for seniors.

Michigan researchers reported on the complexity and difficulty of just creating tools to measure PCMH assumptions — “13 functional domains with 128 capabilities within those domains.” Despite the fact that the whole goal of PCMH is standardization, they dispiritedly conclude: “a one-size-fits-all approach may not be appropriate.” PCMH standardization, tools and accreditation also do not, apparently, obliterate racial/ethnic differences in care.  A study of 1,457 adults receiving care from 89 medical providers within a PCMH-designated practice documented “racial differences in [both] processes and intermediate outcomes of diabetes care …” Another group concludes that after 15 years of NIH-funded projects “primary care transformation is hard work.”

In the face of such underwhelming results, after all the money and effort invested in a bureaucracy that does not contribute to actual patient care, several proponents (including the authors of the New York State summary) suggest that PCMHs just need a little more time to get established. Geisinger Health System touts its proprietary “advanced model” of PCMH. But even they state that, when it comes to better patient outcomes “there is only limited evidence regarding the ability of PCMHs to achieve this goal.” In terms of cost, their analysis of their own product shows “a longer period of … exposure was significantly associated with lower total cost.” However, their calculated return on investment was grimly low, with a large confidence interval range.

Finally, and most importantly, where is the patient experience in all of this? If PCMHs do not clearly improve outcomes and cannot be consistently shown to decrease cost (and may actually increase preventable cost), do they at least make things more patient centered?

Unfortunately, the answer appears to be no. As the evaluators of the 2-year National Demonstratin Project put it, “highly motivated practices can implement many components of the PCMH in 2 years, but apparently at a cost of diminishing the patient’s experience of care.”

Jan Gurley is an internal medicine physician who blogs at Doc Gurley.

Comments are moderated before they are published. Please read the comment policy.

  • LeoHolmMD

    Thank you for what you are doing. Read any Al Lewis?

  • Deceased MD

    Thank you as well. But to be honest, I don’t think we can fight this battle with facts— as absurd as that sounds. Clearly there a lot of dysfunctional things including high pt satisfaction scores related to increased morbidity and mortality. Big corporations don’t care about facts which now include the AMA, hospitals etc. All they care about is $$$$$$$$$$$$$$$$$$$$

  • glasshospital

    As patients have been known to say, “Why do you want to put me in a home???”

  • Shirie Leng, MD

    It’s that way with ACOs as well. I call it “policy before data”. All these things that seem like good ideas on paper get implemented before there is any proof of efficacy. Actually, this jumping the gun tends to happen in all medical policy areas.

    • Dr. Drake Ramoray

      Who thought this was a good idea on paper? I didn’t? Did you? I don’t know any docs who see patients more than one full day a week that thought this was a good idea. Perhaps the ACP, AAFP, and CMS thought it was a good idea. Convinced that it’s coming and you “have to adapt to the times” or coerced into it by their hospital employer sure, good idea? No

      • NPPCP

        So true Dr. Drake. I had a family practice resident come to my clinic to get data about opening a practice in another town. They are pushing PCMH so hard now that these students almost know nothing else. It seems the next generation of physicians are “trapped in medical home nirvana.” They are clueless. I am so sad about this as we need them to stand up and be business owners and starters. I can do it and make a fine living – so can they. It’s a farce, a charade, a mirage.

        • LeoHolmMD

          It’s disgusting. So much time is being wasted on process. Huge interference in learning actual medicine. Residents are going to have to avoid programs with PCMH so they might have medical skills at the end of it. If anyone sets up a DPC program, it’s going to explode.

          • NPPCP

            I admire family practice physicians so much for all of their knowledge and dedication. I just don’t know how they are going to be jerked up out of the rabbit hole. As a Nurse Practitioner, I understand what I do doesn’t replace what they do. Some may not believe that – but it’s true. I love being independent and am grateful for my lot in life but we can’t make it without the family practice folks. They need to just go open their own clinics and start seeing people. It will all work out if they do.

  • southerndoc1

    Big new study in JAMA Feb. 26:
    “A multipayer medical home pilot, in which participating practices adopted new structural capabilities and received NCQA certification, was associated with limited improvements in quality and was not associated with reductions in utilization of hospital, emergency department, or ambulatory care services, or total costs, over 3 years.”
    Color me surprised.

    • LeoHolmMD

      Real shocker, huh? I hope they print enough journals for all the noses that need to be rubbed in it.

  • guest

    I am not sure why this is a surprise. Medicine these days is overrun with “thought leaders” aka physicians who decided that full-time clinical practice was not for them. The more of those folks you have, the more policies and regulatory complications there are, and the more money gets diverted into administration and away from the people who are actually doing the work of caring for patients.

    Too many chiefs and not enough Indians…

  • PrimaryCareDoc

    Good. Can we please put this terrible, costly idea to rest now?

  • W Joseph Ketcherside, MD

    So yet another study showing that rearranging the deck chairs on the Titanic doesn’t provide any better outcomes for patients. Maybe the underlying fee-for-service system plays a role? We are still faced with the highest costs in the world – by a long shot – and 37th-best quality.

    I don’t claim to know what the right answer is, but it most assuredly is not to continue the current dysfunctional mess we euphemistically call a “health system”. It’s good that we have studies like this to show us when one new ideas is a bust so we can resume the search for a better way.

    You know, every other developed nation in the world has figured out how to deliver good health care to all of their citizens. Wonder if we should take a serious look at how some of them do it? Seems to work for them.

    • Dr. Drake Ramoray

      “Maybe the underlying fee-for-service system plays a role?”


      Canada, France, Germany, Switzerland, Australia and New Zealand are all fee for service or at least partially fee for service. The private portion of NHS is fee for service.

      “We are still faced with the highest costs in the world”

      True. But physician payments are a small portion of this problem as if you paid doctors nothing you would reduce health expenditures by about 10%.

      Pharma, device manufacture’s, and hospitals charge way more in the US compared ot other countries. Doctors are cogs (granted relatively well compensated cogs) in the coporate healthcare machine. Recently, more new doctors for the the first time ever will work for a hospital or large medical group (an intended consequenc of the PCMH/ACO movement). This will only drive up costs further and profits into the hands of pharma, device manufactures, hospitals, and now EHR vendors. Let me introduce you to the facility fee.

      “and 37th-best quality.”


      These studies or groups always base this on survival rates from birth and are a poor reflection of the quality of the American healthcare treatments. We have the most overweight people, worst diet, more guns, and more cars, than any other society in the world. The last two in particular have a nasty way of ending an otherwise healthy peron’s life rather abruptly. But when you look at true measures of healthcare, such as cancer survival rates the US beats just about every other country in almost every type of cancer.

      • W Joseph Ketcherside, MD

        Dr. Ramoray is completely correct, it’s not just physician pay. The system as a whole is problematic.

        And yes, the facility fee is total nonsense the way it’s being used to drive up the cost of the same office visit you had before the big health system bought up your doctor’s practice.

        Our quality rating also reflects access, where the US trails all other developed countries. There is no question that an individual patient MAY receive the best care in the world here in the US – but looking across our population, we can’t say that is true for a large number of people.

        Focusing only on how we pay physicians won’t solve the problem. And as I said, I don’t know the answer. If you look across other developed nations, there are several different models that all seem to work. Some have more government control, and others actually have less than the US. But all ensure access to care for all.

        Believe it or not, I’m actually pretty conservative and business-oriented. I think that the best way for our businesses and our nation to be competitive is to ensure that we have a well-educated and healthy work force. Easier said than done, but critical to our continued economic leadership.

        So, we have to keep trying. Studies like this one are very helpful.

        • Margalit Gur-Arie

          The only thing those other countries have in common is that universal health insurance is paid from tax revenue, and nobody profits from basic health insurance.
          So yes, we know what needs to be done, but we choose not to do that.

          • Dr. Drake Ramoray

            They all have less lawyers too.


            And better legal processes for handling malpractice.



            Which is also of course more expensive here.

            “U.S. litigation costs overall are at least twice those in other developed countries, such as Canada and much of Europe, according to a 2008 study by the Manhattan Institute’s Center for Legal Policy.”


          • Margalit Gur-Arie

            You are right, there are other differences, although I’m not sure if they are as important, or that they don’t somehow correlate with the state of mind that leads the people in those other countries to offer tax supported health care.

            Immigration for example, where poor people come in to work hard, should cause those who came before them to move up, getter better jobs and more money, so I am not sure that Milton Friedman was right back then, but I am certain he is right now, because there is no room for anybody to move up.
            So another difference that is pertinent here is the huge inequality in this country that leaves less money in the pockets of most taxpayers, hence the primal opposition to pooling unnecessarily scarce resources.

          • Dr. Drake Ramoray

            There is definately a difference in the state of mind for tax supported healthcare. We only here the horror stories of Canada and NHS not the relatively well functioning systems in Australia, NZ, Germany, and Switzerland in particular. I think they are big differences. Paid for education but lesser compensation, less fear of lawsuits, and immigration.


            Immigration for example, where poor people come in to work hard, should cause those who came before them to move up, getter better jobs and more money

            No disagreement there whatsover. It is well documented that upward mobility in this country is getting much worse. The point I’m trying to make about what Milton Friedman says that if you allow low skilled, low functioning, illiterate people to flood your borders then it becomes much more difficult to support a welfare state based on general taxation.

            I looked into emigrating to Australia or New Zealand once. Applicants to emigrate to Australia are evaluated on a point system that gives preference to young people, English proficiency, education and a useful skill or skill level. Basically they want to know if you are good for Australia and make sure you won’t drain public resources before they allow you to emigrate. Switzerland (looked there too) actually requires you to have family or contact with a current citizen who will vouche for you in addition to requirements similar to Australia.

            The US has a history of relatively open borders “Give me your tired, your poor, Your huddled masses yearning to breathe free….” Emma Lazuras. The idea is that you come here to make it on your own with minimal government assistance.
            Let’s say we move to single payer. So you live here and get tax payer funded healthcare. Would you propose we continue our relatively loose imigration policies or change to a more strict system like Europe and the countries I have cited? Continue to let industries in the US import cheap unskilled labor?

          • Margalit Gur-Arie

            Well, if we continue to obstruct upwards mobility, then having massive poor and unskilled immigration is not sustainable. Lots of other things become unsustainable as well, such as paying for one’s doctor.
            However if we reform our way back to a more balanced society, where hard work is the recipe for success, not just a lottery system for one in a million people, I think immigration of all kinds can keep the country young, vibrant and creative. The first generation toils its life away, and the second generation cures disease. Who knows what the third one can accomplish…

          • Dr. Drake Ramoray

            You are correct there are a whole lot of things becoming unsustainable in this country. Like the other posters have indicated , there are no easy answers, I don’t have them, and we have moved well beyond the scope of original article. As always, a pleasure Margalit

          • Margalit Gur-Arie

            Pleasure is all mine. Thank you.

          • Deceased MD

            As far as being sued it is really dependent on what state you live in. Many states With $250,000 max for “pain and suffering” even if a pt dies, it is hard for family to collect unless the pt that supposedly died from bad medical care was an economic generator for their family. In that case, they can sue away for economic damages but not really for the pain . But a poor housewife really can’t sue in states like this. I spoke with a malpractice attorney once and he sort of abashedly agreed.

          • Dr. Drake Ramoray

            True it varies by state. And man do I need to get in the expert witness business.

Most Popular