Why quality did not improve with hospital EHR implementation

The 2009 Health Information Technology for Economic and Clinical Health Act (HITECH) authorized incentive payments, potentially totaling some $27 billion over ten years, to clinicians and hospitals when they implement electronic health records in such a way as to achieve “meaningful use,” in terms of advances in health care processes and outcomes.

But, are EHRs really “meaningfully useful” or are they more likely to be costly and ineffective?

The latter seems to be one possible interpretation of a recent RAND study of EHR adoption in US hospitals.

The RAND study statistics are impressive: five study authors tallied 17 “quality measures” for three medical conditions against three possible levels of EHR capability (no EHR, basic EHR, advanced EHR) for more than two thousand hospitals for each of 2003 and 2007. They then related changes in quality over the four year timeframe against changes in EHR status (for example, from no EHR to an advanced EHR).

The reported results were disappointing to EHR proponents. Among the hospitals whose EHR capability remained unchanged over the four years, there was no statistically measurable difference in quality improvement between hospitals with EHR capability and those without. For hospitals which upgraded their EHR capability, the performance improvement was generally less than for those who didn’t change, including those with no EHR at all.

So, should we forget about EHRs? Maybe defund HITECH?

Not necessarily.

As the study’s authors point out, there are a several possible explanations for their results other than ineffectiveness of EHRs. Implementation of an EHR—a very demanding effort—might temporarily disrupt other quality improvement efforts. Hospitals with EHRs typically had higher quality measures to begin with, and—like trying to catch up with the speed of light—would likely find improving quality more challenging as 100 percent quality is approached. Results might have been different for other medical conditions. And the timeframe of the study may have been inadequate to measure the impact of new EHRs, some of which may have been implemented only just before the end of the time period.

It can also be argued that the measurement methodology was flawed. Using simplistic indicators of quality like whether or not aspirin was dispensed on arrival or discharge instructions were provided is a little like judging the quality of a meal by whether or not there was a caterpillar in the salad. Presence of a caterpillar definitely indicates a problem, but its absence says nothing about other aspects of the meal. The study authors indicate their awareness of this limitation in stating “we are concerned that the standard methods for measuring hospital quality will not be appropriate for measuring the clinical effects of EHR adoption.”

Perhaps most importantly, as with other IT systems, EHR success depends on the competence of the implementers and the willingness of the users to accept change, with poorly managed projects more likely to foul up existing processes than improve them. The RAND authors praise programs initiated by the Office of the National Coordinator for Health Information Technology to improve EHR implementation, and comment—in spite of the inconclusive results of their study—that “We believe that these programs are well conceived and anticipate that they will lead to more effective use of EHRs, which will in turn lead to improved quality in US hospitals.”

EHR systems are no panacea, and clearly there have been both successful and troubled EHR implementations. What is needed now is a closer look at what works and what doesn’t, how well EHRs perform over a longer timeframe than the RAND study, and a much less simplistic look at what is really happening to clinical quality as a result.

Roger Collier is a consultant specializing in health care policy issues who blogs at Health Care Reform Update.

Submit a guest post and be heard on social media’s leading physician voice.

Comments are moderated before they are published. Please read the comment policy.

  • pcp

    From the original study:

    “We believe that these programs are well conceived and anticipate that they will lead to more effective use of EHRs, which will in turn lead to improved quality in US hospitals.”

    Ah yes. When the data doesn’t support your pre-determined conclusion, tell us what you “believe.”

    Very scientific.

  • BladeDoc

    Or it could be that EHRs are designed and built with the overriding purpose of improving billing compliance and surviving an audit. Any useful information transmitted by EHR notes has, in my experience, been at best purely incidental (if not accidental).

  • http://onhealthtech.blogspot.com/ Margalit Gur-Arie

    Just buying a food processor is not going to provide you with a better, faster, cheaper dinner and it won’t make you a better cook either.
    Just buying an EHR and putting it in place should not be expected to lead to quality improvements. Why should it?

    If the institution has a goal and a plan to improve quality in specific ways, an EHR tool may be able to help execution. EHRs do not dispense aspirin. They can remind you to dispense aspirin, if you configure them that way, which implies that you have a goal of dispensing more aspirin.

    • pcp

      Agree. Isn’t it fair to say that, in many instances, using an EMR has become the goal, not a tool for reaching a more significant goal?

      • http://onhealthtech.blogspot.com/ Margalit Gur-Arie

        I think the recent mad rush to collect incentives is certainly pointing that way. It’s almost like government is telling folks to just get the EHR in there, and we’ll figure out later what to use it for. So maybe we should wait until they decide what they want to do with it before we perform quality studies.
        Perhaps a better way would have been to tell hospitals what the expectations are in terms of quality, including appropriate carrots and sticks, and let them figure out ways to deliver.

  • Leo Holm MD

    Is anyone aware of any serious clinical comparative evaluation of one EHR vs. another?

    It seems that currently, they are all being lumped together when evaluated. How would it be possible to tell if one system is better than another? Are there any EHR systems that are actually detrimental to the quality of patient care?

    • http://onhealthtech.blogspot.com/ Margalit Gur-Arie

      I have not come across anything like that, and I am very skeptical of the feasibility of undertaking such study for many reasons.
      The first and foremost would be that EHR vendors would not be too keen, to put it mildly, on having their products evaluated, and they usually have contractual ability to block this type of research.
      The second problem is that the number of variables that need to be controlled is staggering, particularly in hospital settings.
      Most hospitals employ multiple products from multiple vendors and the definition of what constitutes an EHR is rather elusive. I doubt that you can locate two hospitals with the same exact combination of products. Even for those who have the vast majority of software from a single vendor, each product allows hundreds of various configurations, and larger institutions tend to build their own proprietary customizations. Obtaining a large enough sample, that can be properly controlled, for each EHR will be a daunting task.

      There are also many factors outside the product per se that may affect clinical performance of a given EHR. For example, the amount and timing of training, the implementation timeline, the quality of the particular vendor resources assigned to the project, the quality and redundancy of hardware and the institution’s clinical and non-clinical staffing models are all variables that will need to be considered in addition to the customary patient mix, geography, financials, etc.
      I think it may be somewhat easier to conduct an evaluation in ambulatory settings, but I have not seen that done either.

      Many times you see studies, usually self congratulatory, from various health systems describing quality improvements they were able to achieve. You can usually find out what EHR they are using. Other times there are regrettable incidents reported at this or that hospital. You can also find out what EHR they have. This is the closest we can come to evaluating clinical adequacy of particular EHRs, and this of course it is not a very useful or scientific method.

      • horseshrink

        As usual, I enjoy reading your level-headed posts.

  • buzzkillersmith

    I was at Primary Care Update in Spokane last weekend and the lunch speaker was Dr. Glen Stream, president-elect of the AAFP. His take on EHRs was that fighting them is futile. He did not we should adopt them because they save money or improve pt care. They are just inevitable because it has been decided that they are. I do not disagree with his analysis.
    I believe that EHRs make very little clinical sense but represent money to powerful vendors, who have intellectually and/or financially captured prominent physicians, including Obama administration officials. The data simply do not matter on this issue. Practicing physicians must try to do what we can to minimize the damage of this inappropriate technology.

    • pcp

      You wouldn’t expect the AAFP to go against the interests of their constituency (government agencies and large insurers), would you? They’ve basically been a marketing arm of the EHR industry for the past decade.

  • horseshrink

    I remain convinced that the government’s role in EHR development should be limited to development/maintenance of data standards. I think this is more useful for patient care than flogging and coaxing clinicians with sticks and carrots – to adopt premature technologies that don’t yet fit well with normal clinical patient flow.

    Data standards help HIE, and could make it easier for docs to change products. Currently, the idiosyncracies of product specific data constructs make it too hard to change products.

    While such client-lock might make vendors happy right now, I think it is likely to backfire.

    If I cannot afford to migrate my patient records from one abysmal vendor’s product to another, I’ll just dump to paper and be done with it.

    If the EHR industry wants a durable client base beyond the ephemerality of political projects du jour (like HITECH), they will seek to marry clinicians to the technology … not just a vendor.

    • http://onhealthtech.blogspot.com/ Margalit Gur-Arie

      Completely agree….

    • IanMc

      I couldn’t agree more. In the UK we have a major programme (£13bn;$21bn) which is also going nowhere because it concentrated on trying to deliver systems rather than providing standards for vendors and hospitals to work to.

Most Popular