Radiologists who cheat on their board exams: Who’s to blame?

Radiologists who cheat on their board exams: Whos to blame?In a widely circulated CNN article, many radiologists have been found to cheat on their board exams: “Doctors around the country taking an exam to become board certified in radiology have cheated by memorizing test questions, creating sophisticated banks of what are known as ‘recalls,’ … The recall exams are meticulously compiled by radiology residents, who write down the questions after taking the test, in radiology programs around the country, including some of the most prestigious programs in the U.S.”

The American Board of Internal Medicine faced a similar issue recently, as doctors got into trouble for using a board preparation course that used “recalled” test questions.

What’s happening with the radiologists, however, seems more egregious. Apparently, these large banks of questions were considered “prized possessions” of radiology residencies. In some cases, residents were encouraged by their program directors to use the illicit resource:

Webb, 31, said he failed the first radiology written exam, which focuses on physics, in the fall of 2008. He said the program director at the time, Dr. Liem Mansfield, told him to use the recalls in order to pass.

“He told me that if you want to pass the ABR physics exam, you absolutely have to use the recalls,” Webb said. “And I told him, ‘Sir I believe that is cheating. I don’t believe in that. I can do it on my own.’ He then went on to tell me, ‘you have to use the recalls,’ almost as if it was a direct order from a superior officer in the military.”

Thankfully, this appears to be largely confined to radiology. I’m not aware of such a bank of questions in internal medicine, or any other specialty.

The American Board of Radiology, however, may have unwittingly fostered this underground culture of recalled test questions. Apparently, half of the questions on their certification exam were the same from year to year.  Is it any wonder why, then, old test questions were treated like gold?

Do other Boards use such a high proportion of repeat test questions? I’m not sure, but 50% sure sounds high.

I understand that composing high stakes board certification test questions is an exhaustive process, but if the Board really wanted to stamp out the culture of recall, simply ensuring that the board questions were all new would have been the easiest way to do so.

Had they taken that pro-active approach, the Board may have spared the entire field of radiology this black eye.

Shortly after this post was published, I received an email from Donna Breckenridge, Director of Communications and Editorial Services, The American Board of Radiology. With the Board’s permission, her email is printed below:

Dr. Pho – We saw your comments on your blog today and wanted to call your attention to a “fact” that you apparently picked up from CNN but that is wrong. Despite our discussing this with CNN multiple times, they reported that half of the questions on our exams are the same from year to year. In fact, the opposite is true and every year’s test is different. Half of the questions are brand new. The other half are drawn from a question bank of thousands of previously used questions. These questions are never the same as the ones used on the most recent exam, and they can be any combination of those thousands of previously vetted questions.

Drawing from the bank of previously used questions allows for the required statistical reliability to validate each examination. Without reused questions, the difficulty level of each new exam could vary in unknown ways from year to year. Additionally, an unknown number of the completely new test questions may perform poorly on the new examination, necessitating their removal from scoring. This could result in an unbalanced, nonequivalent examination. For these reasons, best testing practices typically incorporate the re-use of questions that are known to perform well.

I’d encourage you to look for further information on the ABR’s Exam Security Policy, and on the ABR’s exam creation process. Please see the Announcements section of the ABR’s website and “The Truth about the ABR’s Examinations.”

 is an internal medicine physician and on the Board of Contributors at USA Today.  He is founder and editor of KevinMD.com, also on FacebookTwitterGoogle+, and LinkedIn.

Comments are moderated before they are published. Please read the comment policy.

  • http://pulse.yahoo.com/_QSINMKGBLNT7HW3SF37JPWABQM Alexander

    I am a current radiology resident, and I hope I can help to clarify this very sensationalized story. The road to board certification in radiology requires 3 examinations: The physics exam (usually taken at the beginning PGY-3), the written exam (usually taken at the beginning of PGY-4), and the oral exam (taken at the end of PGY-5, the last year of residency).

    The written and physics exams are the steps for which banks of ‘recalls’ are collected. It is widely agreed upon by trainees and training programs that there is a very high proportion of irrelevant and esoteric material on these examinations; questions that have little or nothing to do with the practice of radiology. Additionally, there is no material published by the American Board of Radiology (ABR) to guide the trainee in preparation for these exams. These unfortunate conditions necessitate the use of recalls in order to pass the exams. The reality, however, is that nearly all standardized exams use recalls, the difference is that for other exams, it is provided by companies and the trainee must pay for the questions. How do you think Kaplan and The Princeton Review generate their ‘practice questions’ for the SATs, MCAT, GRE, etc. . . That’s right, by people taking the exam and remembering what questions and topics were covered.

    The recalls themselves are used as a study aid, and are not merely memorized by the test taker as the CNN article implies. Because these recalls are generated by trainees (both the questions, the answer choices, and the explanations), they are fraught with errors. It is, therefore, up to the trainee to look up and read about the subject being tested, to ensure that the answer which a previous trainee believed was correct, is indeed, the correct answer. Additionally, the questions are often changed slightly from year to year, such that changing one word in the question stem will make a different answer choice correct.

    Furthermore, there are thousands (likely tens of thousands) recall questions available for a ~200 item examination. Studying this many questions requires an understanding of the material, not simply memorizing a few questions, then passing the exam, as suggested by the article.

    But in truth, the physics and written examinations are not very important in the process of becoming board certified. These exams require are few weeks of preparation, followed by a 2 hour exam. The “real” board examination is the oral boards: a test in which every 4th year radiology resident travels to Louisville, Kentucky, where a hotel is taken over by the examination. Examinees go from room to room where they are shown images, and must discuss the case in terms of findings, differential diagnosis, and further management or imaging decisions that need to happen. The examiner is free to ask any question that he/she feels appropriate to evaluate the examinees level of knowledge. These are not multiple choice questions which can be memorized or learned from a recall. This is a very anxiety provoking exam, and residents spend about 6 months of long and high-intensity studying for it.

    Of course, the CNN article made no mention of the oral examination component of radiology board certification, because such a discussion would reveal that their story is of trivial significance. I felt that this story irresponsible reporting, and published to generate a scandal where there is none.

    • http://www.facebook.com/jonathan.marcus.ca Jonathan Marcus

      I’m a family doctor in Toronto.  I figured that the physics exams was likely on fairly esoteric and irrelevant material.  Natural that residents would want to get by this stuff quickly.  Though it’s too bad that things that residents are expected to understand don’t correspond with what’s on the exam.  Perhaps the educational bodies should look at standardizing the learning in physics and write exams that are reasonable.

  • http://pulse.yahoo.com/_2LRZNHDZS6DU45WQ567LPQ7CMI ninguem

    Is that cheating? I wonder.

    The American Board of Whatever has determined that you need to know A, B, C, D, and E to be a certified whateverologist. So the candidates studied up on A, B, C, D, and E. The specialties with oral exams have a limited number of scenarios that you can possibly ask a candidate.

    So……you practice on the subjects the Board has determined you need to know.

    As opposed to guessing what the Board wants you to know.

    They’re not describing a cheat where you memorize that the correct answer to Question 1 is “C” as they rotate the questions, the distractions, and the order. At least I hope they do.

    The subject…..any medical specialty…..is infinite. Is it unfair to recognize the more limited basic knowledge set you need to consider yourself a competent internist or surgeon or radiologist?

    You’re taught all about mechanical engineering and paint chemistry and seat belt technology and the basics you know to drive a car get lost in the shuffle.

    Stanley Kaplan took the Mickey out of the SAT’s and similar, when the mysticism considered the exams some sort of academic DNA that can’t be changed. Yes it could be changed.

    And when I see the bloat and complexity added to my own boards…….the astronomical salaries for the “leaders” and the sky-high fees for these exams……to no good end…..I have absolutely, positively, zero sympathy for the Boards.

    “Recall” away, as far as I’m concerned. There’s a competing certification organization, the AAPS, that’s been around since 1950. They exist for a reason, and I hope they expand their scope.

  • http://pulse.yahoo.com/_2LRZNHDZS6DU45WQ567LPQ7CMI ninguem

    Is that cheating? I wonder.

    The American Board of Whatever has determined that you need to know A, B, C, D, and E to be a certified whateverologist. So the candidates studied up on A, B, C, D, and E. The specialties with oral exams have a limited number of scenarios that you can possibly ask a candidate.

    So……you practice on the subjects the Board has determined you need to know.

    As opposed to guessing what the Board wants you to know.

    They’re not describing a cheat where you memorize that the correct answer to Question 1 is “C” as they rotate the questions, the distractions, and the order. At least I hope they do.

    The subject…..any medical specialty…..is infinite. Is it unfair to recognize the more limited basic knowledge set you need to consider yourself a competent internist or surgeon or radiologist?

    You’re taught all about mechanical engineering and paint chemistry and seat belt technology and the basics you know to drive a car get lost in the shuffle.

    Stanley Kaplan took the Mickey out of the SAT’s and similar, when the mysticism considered the exams some sort of academic DNA that can’t be changed. Yes it could be changed.

    And when I see the bloat and complexity added to my own boards…….the astronomical salaries for the “leaders” and the sky-high fees for these exams……to no good end…..I have absolutely, positively, zero sympathy for the Boards.

    “Recall” away, as far as I’m concerned. There’s a competing certification organization, the AAPS, that’s been around since 1950. They exist for a reason, and I hope they expand their scope.

  • http://thehappyhospitalist.blogspot.com happyhospitalist

    I guess the 80 hour resident work week didn’t result in more studying.

  • http://www.facebook.com/profile.php?id=1404048836 David Behar

    These exams are racially biased, and have no scientific validation. They are rent seeking, exclusionary, anti-scientific garbage. One almost has a duty to recall the questions. Furthermore, the test makers are too lazy to write entirely new questions, they are begging for recall. As usual, left wing extremist and New Englander, Dr. Pho will bash the clinician, rather than go up against invalid authority.

  • Anonymous

    Dr.Pho, either you are the only practicing physician in the solar system to never have used study aids, including the dreaded practice questions in his academic career or you are a hypocrite. As a board certified radiologist I can tell you that the written board exam is 95% esoteric B.S. and would be near impossible to pass without a guideline(i.e. past questions). The oral exam on the other hand is not only extremely difficult but is  very real world. I can assure you that the vast majority of radiologists are fairly bright and are not cheaters by nature. The problem is not as egregious as you think.

  • David Price

    In a time where blame is always assigned to a larger organization it is important to remember that the choice to provide recalled questions to the bank or to use questions from the bank is made by the individual.

  • Anonymous

    Back in the 1950′s my classes used to study from AMSCO Review books that had several previous New York State Regents exams. We started going over old Chemistry exams just after Christmas break. It would be cheating if only a small percentage of students used the books. It becomes essential when almost everybody does it. This is the property of “equilibrium” in economics.

    Going over old tests, when combined with other instructional techniques, is a learning experience too.

    Maybe some questions about dose and response should be included in medical exams. Pediatricians want to ban bicycles for children because brass is used in the air valves. That level of idiocy should be investigated. 

     

  • http://twitter.com/spinesurgeon Bruno Mascarenhas

    Going through the previous year’s questions and preparing answer for them, is a part of test preparation

    I do not accept that this is cheating

    “Cheating” is getting the present year question paper before the exam :) :) (through some crooked means) and preparing answers for that !!

    • http://pulse.yahoo.com/_2LRZNHDZS6DU45WQ567LPQ7CMI ninguem

      Bruno…..”I do not accept that this is cheating”

      Agreed. And given the behavior of the Boards recently, across all the specialties, they do NOT get the benefit of the doubt from me.

Most Popular