Now, as you know, I am a big fan of epidemiology. I do not believe that a randomized controlled trial is the be-all-and-end-all in evidence generation, and a well done observational study can add to our reservoir of knowledge much more efficiently. Of course, I, as many others, acknowledge certain limitations of epidemiologic design. However, many of them can be overcome with careful design and analysis.

I have to confess, though, that over the last week I have bumped into two news stories that have made me cringe. The first, based on a Kaiser study, showed that people “drinking at least four cups of coffee a day were 18 percent less likely to be admitted with a heart rhythm disturbance than those who drank no coffee at all.”

So, great, the public may take away the message that drinking coffee prevents a-fib. Well, I have to say that the reporting of this was measured and tried to avoid this unfortunate inference of causality. Yet, it left enough room to imply that yes, perhaps there is a causal link. So, what’s wrong with that?

Bear with me while I bring in the second example of a study that bugged me, based on the Women’s Health Initiative. You may recall that the WHI is the large NIH-sponsored study that a few years ago turned hormone replacement therapy on its head. The study had a randomized component and an observational component. So, the newest analysis shows that women “who drank the equivalent of one to two drinks a day — be it beer, wine, or liquor — were 30% less likely than non-drinkers to become overweight or obese.”

Do you see the similarities? So why am I bothered? To me this is the classic case of a high potential for confounding by indication. What’s that you say? That is a situation in which a subject that has the exposure in question (in these two cases coffee and alcohol) is inherently different from one who does not, and this difference is what determines the probability of the exposure itself. Why should this present a problem in a study where the authors carefully adjusted for confounding, which is true for both of the studies? It is a problem because the kind of confounding that this represents is impossible to tease out without real-time attention to the subject.

Here is how it would work in the case of coffee study. Say I am a person with paroxysmal (occasional) a-fib, and I have noticed that if I drink so many cups of coffee per day, I get into brief episodes of palpitations. Not enough to send me to the doctor’s office or the hospital, but enough to start thinking about cutting out caffeine. So, I stop drinking caffeine, and continue with my baseline frequency of a-fib attacks.

You see the problem? Is it possible (or even probable) that those people who drink four or more cups of coffee per day somehow have an inherently higher threshold for slipping into their a-fib than those who do not? And if the answer is “yes”, then the four cups become a marker for someone who can tolerate them, rather than the cure for a-fib. You can construct a similar explanation with the two drinks and weight.

So, while I love epidemiology and its methods, I am wary of hanging my hat on associations that may likely be explained by confounding by indication. And although the stories were reported with many caveats, human nature may prevent us from hearing the nuance. It is clear that in both these instances the burden of proof is on the researchers to show me that I am wrong.

Marya Zilberberg is founder and CEO of EviMed Research Group and blogs at Healthcare, etc.

Submit a guest post and be heard.