Do surgical checklists work? A closer look at the data.

The other day Atul Gawande tweeted the following:

I am not against checklists. When I was a surgical chairman, I implemented and used one in both the operating room and the ICU. They do not add costs and may be helpful.

However, the randomized trial that Gawande referred to does not necessarily settle the issue about whether checklists really do reduce complications and deaths.

The paper, published online in Annals of Surgery, looked at 5,295 operations done in two Norwegian hospitals. The intervention was a 20-item checklist consisting of three critical steps: the sign in before anesthesia, the timeout before the operation began, and the sign out before the surgeon left the operating room. Using a stepped wedge cluster design, patients were randomized to control or the checklist.

Complications occurred in 19.9% of the control patients and 11.5% in those who got the checklist, a significant difference with p < 0.001.

A look at table 2 of the study finds that of 27 complications or groups of complications, 14 occurred in significantly fewer patients in the checklist group.

Of the significant 14, a few, such as cardiac or mechanical implant complications, could possibly have been prevented by the implementation of the checklist.

For most of the others, the relationship between the use of a checklist and a post-operative complication is tenuous. How could a checklist possibly prevent technical complications like bleeding requiring transfusion, surgical wound dehiscence, and unintended punctures or lacerations?

Here are a few more of the complications that occurred significantly less frequently in the checklist cohort—urinary tract infection, pneumonia, asthma, pleural effusion, dyspnea, and the nebulous categories of “complications after surgical and medical procedures” and “complications to surgery not classified.”

What item on a checklist prevents asthma, UTI or any of those on that list?

Embolism, sepsis, and surgical site infection, three complications one would expect a checklist to impact because of reminders to give prophylactic antibiotics and anticoagulation, did not occur at significantly lower rates in the checklist group.

Even the cardiac complication category is open to question because none of the 5 subcategories (cardiac arrest, arrhythmia, congestive heart failure, acute myocardial infarction) differed significantly between the two groups. Only when the 5 were combined did statistical significance emerge.

In the 300-bed community hospital, checklist use was associated with a significantly lower mortality rate than non-use, 0.2% vs. 1.9% respectively (p = 0.02), but no mortality difference was seen in the 1100-bed tertiary care hospital.

The tertiary care institution enrolled 3,811 patients while the 300-bed hospital contributed 1,083. If more patients had been in the latter group, the difference may have disappeared due to the principle of regression to the mean.

Despite the heightened vigilance associated with an ongoing research project, compliance with checklist use was only 73.4%.

Before you go off on me, I will remind you that I do not oppose checklists. Most things we do in medicine are not based on class 1 evidence.

Just don’t tell me that checklists have been proven to reduce complication rates or save lives.

“Skeptical Scalpel” is a surgeon blogs at his self-titled site, Skeptical Scalpel.

Comments are moderated before they are published. Please read the comment policy.

  • JR DNR

    Aren’t checklists all about Cognitive Load? The brain is only capable of consciously remembering so much information at any time. By providing a checklist, this allows the team to delegate remembering those tasks to the checklist, and then focus their attention on other things. This comes from Cognitive Psychology research.

    Further Reading (Cognitive Psychology applied to User Interfaces or Speakers/Teachers):
    Designing with the Mind in Mind by Jeff Johnson
    100 Things Every Presenter Needs to Know about People by Susan Weinschenk

  • Shirie Leng, MD

    The comment by JR DNR about cognitive load is a valid one, and one of the reasons skeptical does not object to checklists out of hand. They can be helpful. It is the blanket application of checklists in every situation, no matter what, that is irritating and, frankly, somewhat embarrassing. In addition, such mindless use of a one-size-fits-all checklist diminishes the usefulness of the list, since people tend to tune out things that are repetitive and meaningless. Finally, the checklists contain dubious and seemingly random items. For instance, a time-out in which the question “is the patient in the correct position” after the patient is prepped and draped and the scalpel ready to go, makes no sense at all. Same for making an anesthesiologist do a time-out before doing a labor epidural. “Can you state your name, ma’am” “Arghhhh!” “Is the patient in the correct position?” “Not for long, you idiot, just give me pain relief!”

  • Skeptical Scalpel

    Thanks for the three comments. I agree that one size fits all and having too many items on a checklist are distracting and may lead to the “tuning out” that not only defeats the intended purpose, but may be harmful. I believe they are far less effective and less necessary for emergency procedures where the opportunity to misidentify a patient or operative site is far less.

  • Skeptical Scalpel

    Your approach seems similar to mine. Every 767 jet can have the same checklist, but probably not every patient.

  • Doc c

    The fact that we even consider randomized trials necessary to decide on the value of process in medicine shows how far the academic elites have driven us from the reality of our vocation. Did they do randomized trials in the airline industry? Good habits don’t need to be proven by scientific analysis. And in any case, real science does not accept statistical certainty at the 2 sigma level, as medical science does. Why pretend we know more than we do from RCTs? We’re feeling our way in the dark. Get over it.

    • Skeptical Scalpel

      I’m not sure what your point is. Everyone knew that IPPB was good for postop patients until someone proved it wasn’t. Everyone knew that PA catheters saved lives until someone proved they didn’t. Everyone knows that checklists are great. But what if they divert attention away from something else? What if they impart a false sense of security that everything is OK?

      • Doc c

        My point is how do you blind process? There are too many variables to control in such an experiment. I repeat, did they do a double blind trial of checklists for aviation? No. Those results speak for themselves. You monitor process and outcomes

  • Guest

    My point is how do you blind process? There are too many variables to control in such an experiment. I repeat, did they do a double blind trial of checklists for aviation? No. Those results speak for themselves. You monitor process and outcomes.

Most Popular