Radiologists on the new mammogram study. Who’s right?

The BMJ recently published the latest results from the Canadian National Breast Screening Study (CNBSS). A brief summary of the CNBSS: Women were randomly assigned to annual mammography or breast exams and then the outcomes tracked. The results in the BMJ: mammography did not improve survival.

This is a very interesting study and when I first started working on this post I wanted to delve more into the science of this article and the growing body of literature that is shedding some doubt on the validity of mammography. Doing my due diligence I read the study several times, read the responses that have already appeared on the BMJ site, read some other studies, and read both the Canadian and the American College of Radiology responses. And that’s where I got stuck.

The Canadian radiologists basically said they agreed with the ACR who called the study “incredibly flawed and misleading” (which IMO are fighting words, it’s a bit like saying your study is a piece of garbage and should be ignored). Of course this piqued my interest.

The ACR claim rests on these 3 points:

  • The mammography equipment is old/bad and this was confirmed by independent experts. The lead author, Anthony Miller has refuted this claim in interviews with several Canadian news outlets and an expert from Dartmouth (Dr. Gilbert Welch) calls this study the most meticulously conducted and reported randomized trial on screening mammography. The ACR cites a paper in a radiology journal from 1990 that they say evaluated the equipment and Dr. B. Kopans, a Professor of radiology from Harvard, says he personally reviewed the equipment and found it lacking. How do you deal with these kinds of claims? Maybe ask the radiologists who read the films? It is an important point.
  • Only 32% of cancers were detected by mammography in the study. This is the most interesting from a scientific standpoint and not a “he said/she said” argument. I wanted to write more on this until I saw the last point made by the ACR …
  • Where the ACR basically accuses the authors of misconduct. The ACR statement: “To be valid, randomized, controlled trials (RCT) must employ a system to ensure that the assignment of women to the screening group or the unscreened control group is random. Nothing can/should be known about participants until they have been assigned to one of these groups. The CNBSS violated these fundamental rules. Every woman first had a clinical breast examination by a trained nurse so that they knew which women had breast lumps, many of which were cancers, and which women had large lymph nodes in their armpits many of which indicated advanced cancer. Before assigning the women to be in the group offered screening or the control women, investigators knew who had large incurable cancers. This was a major violation of RCT protocol. It most likely resulted in the statistically significant excess of women with advanced breast cancers assigned to the screening arm compared to those assigned to the control arm. This guaranteed more deaths among the screened women than the control women. The five year survival from breast cancer among women ages 40–49 in Canada in the 1980s was only 75 percent, yet the control women in the CNBSS, who were supposed to reflect the Canadian population at the time, had a greater than 90 percent five year survival. This indicates that cancers may have been shifted from the control arm to the screening arm. Coupling the fundamentally corrupted allocation process …”   (the italics are mine).

However, the exact wording from the BMJ article about the randomization is as follows:

The examiners had no role in the randomisation that followed; this was performed by the study coordinators in each centre. Randomisation was individual and stratified by centre and five year age group. Irrespective of the findings on physical examination, women aged 40-49 were independently and blindly assigned randomly to receive mammography or no mammography.

So the authors are saying their randomization was blinded and the ACR’s counter-claim is that is couldn’t have been. Both can’t be right. The ACR is either accusing the author of lying or saying he had rogue study nurses who didn’t follow protocol. The ACR does not provide any references to support this claim.

This last part of the ACR claim sounds a lot like school yard taunt, “You lied. How do I know? Because I said so.” It is the kind of claim one makes when one, a) hasn’t thought it all through and is letting emotions rule, b) has an otherwise weak argument that needs to be bolstered, or c) you want to be over the top to get page clicks. This accusation actually leads me to evaluate the ACR’s other two claims with greater scrutiny.

I contacted both the Canadian and the American Colleges of Radiology for clarification. I specifically asked the ACR for hard data to back up this 3rd claim. The Canadians said they’d call (they haven’t) and the ACR replied as follows:

Critiques of study design and execution are routinely done for all scientific studies and do not constitute accusations of fraud. However, the irregularities in this trial design and execution raise valid scientific concerns. It is not a matter of whether we, or anyone else, “believe” trial coordinators, but that this is science. The responsibility lies with the trial conductors to demonstrate the soundness of the randomization process. It is not a matter of others in the scientific community to simply “believe.” The trial design and the subsequent outlier results raise these valid questions which are clearly outlined in many peer-reviewed publications cited in our response and elsewhere. We stand behind the statement that this study should not be used to formulate breast cancer screening policy.

For me this statement isn’t enough. If you are a national organization representing physicians you cannot continue to make claims about how subjects were randomized without proof. Saying the patients weren’t randomized appropriately isn’t a study design issue, it’s an ethical conduct issue because the authors say they did randomize blindly. The science goes both ways. In essence the ACR is saying, facts matter but we don’t have to provide any.

If the ACR is right and the equipment was faulty and the science doesn’t support a 32% breast cancer detection rate by mammography why bring up a completely unsubstantiated 3rd claim about a corrupted randomization process? The science should be enough, don’t you think?

It’s food for thought.

Jennifer Gunter is an obstetrician-gynecologist and author of The Preemie Primer. She blogs at her self-titled site, Dr. Jen Gunter.

Comments are moderated before they are published. Please read the comment policy.

  • Markus

    Like Dr. Gunter, my first reaction to the ACR response was that there was an implication of mischief by the investigators. I am glad to see Dr. Kopan’s clarification.
    There have been other reports suggesting that there is not much value to screening mammography for example a longitudinal study of the Swedish experience published in JNCI in 2012. What studies on mammography should we be reading?
    My personal sense is that we ought be trying to clear up the causative factors in breast neoplasia. It is my prejudice that a dollar spent on smoking prevention will save more lives than a dollar spent on lung cancer screening. The drop in breast cancer that happened coincident with a sharp decline in menopausal HRT suggests that here are potential benefits from a better understanding of breast carcinogenesis. I grant that breast cancer seems to be quite heterogenous, and unraveling the process will be arduous.

  • http://www.myheartsisters.org/ Carolyn Thomas

    Food for thought indeed, Dr. G. Thanks so much for this thoughtful and wide-ranging overview of what’s turned into a wee bit of a medical slugfest. “You lied. How do I know? Because I said so.”

    As in all things, we all need to consider the source. Are we really that surprised that organizations representing radiologists (the very ones who have been convincing women of the benefits of mammography screening for decades?) now come out swinging against any study that dares to suggest that such conviction might be unwarranted?

    Reminds me of the recent kaffuffle around new cholesterol guidelines that essentially ditched the need for docs to look at lipid target numbers anymore in favour of overall cardiac risk factors (based on “lack of credible evidence” around reaching those specific LDL targets). What?!? Ignore those LDL numbers? Until that moment, previous studies (at least, those paid for by the folks who make statins) seemed to indicate that lower LDL numbers
    will decrease my risk of suffering a cardiac event, as well
    as death, as well as making my hair shiny and manageable.

    So, how to do a graceful about-face on this issue after decades of training patients that it’s those numbers that matter most? Suddenly, it’s: “Pay no attention to the man behind the curtain!” It may help to explain why Canadian cardiologists are still clinging to their newly-release national cholesterol guidelines that utterly ignore the November U.S. guidelines.

    In both cardiology and radiology, who is an average dull-witted patient to trust?

  • doc99

    Boom.

  • buzzkillersmith

    The study is right. Ignore the rads and ignore the urologists on prostate CA screening to boot.

    Next case.

  • querywoman

    I am 57 years old and have never had a mammogram. In the past ten or fifteen years, I have had occasional physicals that included breast and pelvic exams from internists or family practitioners. They do more on an exam than an ob/gybn.
    I went to a gynecologist right after the Canadian study was publicized. He wants me to have a mammogram so badly because mammograms have found cancers he couldn’t feel. He didn’t answer when I said they might not have needed treatment.
    I’ve read all the letters comments to a NY Times article about, “Vast Study…,” on mammograms that was printed recently. I told him one woman had a fatal stroke after getting a breast cancer dx.
    Women die from lots of other causes all the time, but there is no rush to try and prevent us from dying from them.
    I have no plans to have a mammogram.
    Why is the medical establishment so opposed to criticism?