Should medical schools start teaching to USMLE Step 1?

In the 2012 National Residency Match Program survey, which is sent out to residency program directors around the country by the NRMP, the factor that was ranked highest with regards to criteria considered for receiving an interview — higher than honors in clinical clerkships, higher than extracurricular experiences or AOA election, and even higher than evidence of professionalism, interpersonal skills, and humanistic qualities — was the USMLE Step 1 score.

When considering where to rank an interviewed applicant, the score took a backseat to some of the aforementioned criteria that are perhaps more telling of what kind of person the interviewee is, although it was still one of the highest considered criteria for ranking applicants as well. When a single exam is given this level of importance in determining a future physician’s most critical period in career development — their residency — we have to look carefully at our system.

Two points of consideration come to mind. First, is it wise to weigh a test score so heavily? Many students and faculty could easily point out that student performance on exams by no means always reflects their clinical acumen and social skills when seeing patients. Medicine is, after all, an art far more than a science. Nonetheless, it would be foolish to assume that scores have no worth — a high score on an exam, particularly a behemoth such as the USMLE Step 1, points out many qualities in an individual: hard work, persistence, discipline, and frankly, an understanding of textbook medicine. And thus, we are left somewhere in the middle — perhaps we should weigh scores less than we do, but when you have to sort through thousands of applications, the only standardized metric to quickly compare is, in the end, a number somewhere between 192 and 300.

This brings me to my second point of cogitation — if we are to accept that the system will run with a heavy dependence on a single score, does the said exam test what is clinically essential and do medical schools teach their students what is needed for this exam? As a second year student currently beginning my boards preparation, I know there is a problem here. Most students in my class and around the country rely heavily on additional preparation materials that cumulatively cost nearly $1,000 in preparation for the USMLE. Recently, after completing two years of our clinical curriculum, many of us felt uneasy at best with regards to mastery of boards content when taking a practice NBME exam before our 1 month of allotted study time.

After paying thousands of dollars in tuition, I would expect more. My school, like many top research institutions, prides itself on not teaching to the boards. We have lectures from several world-renowned physicians and scientists, many who present cutting edge clinical and basic science research as part of our curricular program, but many lectures ultimately have superfluous minutiae when framing our educational objectives in the context of the USMLE. As students with an insatiable curiosity and passion for learning, we gladly and earnestly absorb from and appreciate the wisdom of our instructors, but as future applicants to extremely competitive residency programs, I can’t help but wonder if our medical education system is structurally misaligned.

Abraar Karan is a medical student who blogs at Swasthya Mundial.  He can be reached on Twitter @AbraarKaran.  This article originally appeared on The Health Care Blog.

Comments are moderated before they are published. Please read the comment policy.

  • SteveCaley

    The point is well made. Abraham Flexner, in his report in 1910,
    lambasted the medical system present at the turn of the century as
    strictly didactic, without any recourse to practical and experiential
    learning. He stated:
    “Finally, the creation of state boards (of
    licensure) has compelled a greater degree of conscientiousness in
    teaching, though in many places, unfortunately, far too largely the
    conscientiousness of the drillmaster.”
    The American system of
    education has coalesced around the multiple-choice test as some sort of
    verifiable instrument of knowledge. We have bought heavily into the
    mechanistic quantifiable assumptions of testing, with no assurance that
    it is anything more than a self-selecting force. It is very hard to
    teach, and even harder to make sure students learn. The principle of
    “teaching to the USMLE step 1″ elevates the USMLE to a position of
    arbitrary authority over whether and how well a student is taught.
    Homogenization of “learning delivery” has led to ineffective public
    schools, and undergraduate institutions that seem lost on their purpose,
    and even utility for later employment. Can we make medical school even
    more like the Public School System? Survey says, “YES!” Should we?

  • Dave

    First, I think that when you take Step 1 you will be surprised at how much of that “superfluous minutiae” shows up. Boards and shelf exams have a lot more trivia on them than you may realize; the problem is that medicine as a whole has so much of that stuff that any lecture on it would be relatively low-yield as far as boards go. Still, basing a curriculum on a boards review book would be even more insane. Much of the seemingly low-yield stuff will become much more important during clinicals and later in practice.
    Second, there have been a number of papers through the years showing an extremely strong correlation between Step 1 scores and preclinical grades. Yes, there are always a few exceptions, but for the most part the people making straight A’s (or the equivalent) are usually scoring 250+ on their Step 1. All you really need is 2 years of hard work, a good question bank, and 2 weeks of review. If you learn, truly learn, and don’t just memorize/forget during preclinicals, then Step 1 is a breeze. No need to make M1/M2 into one giant boards review course.

  • SteveCaley

    I think that all the USMLE examinations tested things that were not
    principal to the practice of medicine. (I tutored students through the
    Medical School on the USMLE I in third year. I don’t remember my scores
    but they must have been okay.)
    Premed days seemed to train students to assemble a mass of information and correlate it in a rudimentary fashion; and then promptly forget it, as it was generally completely tangential to their long-term intent. An A+ is vital in premed; the name of the course is secondary.
    We train people in “wax-on, wax-off” learning, and regurgitation for the standardized examinations (again, I took the MCAT in 1990 and my lowest individual score was an 11, so this isn’t sour grapes.) We pretend that this stuff is important; often, it is not.

  • Paul Kempen

    The USMLE was created by the FSMB who started with tests for FMGs (ECFMG) in the ’60s. THis progressed to FLEX then USMLE, PT 1, then 2 then 3 at a time when it was important to have an index for schools without regulatory oversight. The “product” was grown and became more expensive as possible to increase the FSMB’s (a private corporation) revenues to around $50 million annually (2011 IRS 990s). This is no longer the case with ACGME for US schools. The continued push for tests of “validation” has been seen by The American Boards Of Medical Specialties (ABMS) and now we find these tests and other “products” are being forced on Doctors by both FSMB (as Maintenence of Licensure=MOL) and ABMS (as MOC). The FSMB MOL is an attempt to move into the Physician marketplace because there is larger numbers, lifelong repetitive opportunity for extortion of fees and ability to charge even more inflated fees from “rich doctors”. There is NO evidence that any improvements in quality of medical care or doctors has occurred. This is regulatory capture of a well paying group of the population. These serve only the “bean counters” who need something to count in numeric scale-because they have no time, interest or ability to really want to evaluate what makes the “best” resident or doctor. Yes, It is well time to critically evaluate these very profitable “non-profit corporations” given the 6 and 7 figure incomes of the CEOs there, well above payment to working physicians. It may be time for a government based agency to administrate any such testing-alone because the corporations simply make decisions without any oversight and voters at least can create pressure beyond that which “the market will BEAR”! SUch testing may well deserve the test of randomized controlled trials to prove it is incompetent or simply be discarded as a relic of the past.

  • Paul Kempen

    Also see these references for details on the regulatory capture of medicine by FSMB and ABMS:

    N Engl J Med. 2013 Mar 7;368(10):889-91. doi: 10.1056/NEJMp1213760.

    The Step 2 Clinical Skills exam–a poor value proposition.

    Lehman EP 4th1, Guercio JR.
    Maintenance of Certification – important and to whom?

    Paul M. Kempen, MD, PhD

Most Popular