Statistical analysis in medical journals should be unbiased

by Todd Neale

Editors of medical journals should unite in requiring independent statistical analyses of industry-sponsored clinical trials, according to the two top editors of the Journal of the American Medical Association.

Such a measure would help prevent any inappropriate influence from the study sponsors on data analysis and reporting, wrote Catherine DeAngelis, MD, MPH, editor-in-chief of JAMA, and Phil Fontanarosa, MD, MBA, executive deputy editor of the journal, in an editorial.

“This approach would add powerful support to the fundamental principle that physicians must first do no harm,” they wrote in an editorial in the March 24/31 issue of JAMA.

Steven Nissen, MD, of the Cleveland Clinic, agreed in a commentary in the same issue.

“Although this procedure does not guarantee the integrity of the resulting manuscript, it is an essential step and should be universally mandated,” he wrote. “If all journals adopted such a policy, the quality of reporting of industry-sponsored clinical trials would be significantly improved.”

Providing a basis for his argument, Nissen recounted the situation surrounding rosiglitazone (Avandia) and the RECORD (Rosiglitazone Evaluated for Cardiac Outcomes and Regulation of Glycemia in Diabetes) trial.

In February, the Senate Finance Committee released a report on its investigation into safety issues surrounding the drug and the role the manufacturer, GlaxoSmithKline, played in the conduct of the RECORD trial and the release of the results.

The report suggested the drug should be pulled from the market. The FDA will convene an advisory panel in July to address the issue.

Nissen became involved in 2007 when he submitted a manuscript co-authored by Kathy Wolski, MPH, also of the Cleveland Clinic, to the New England Journal of Medicine. The meta-analysis of data from 42 clinical trials found a 43% increased relative risk of myocardial infarction with rosiglitazone.

The manuscript was leaked by a peer reviewer to GlaxoSmithKline, whose own statisticians upheld the findings.

The manufacturer then took steps to blunt the effects of the meta-analysis, according to Nissen. The RECORD trial was unblinded and the interim results were published.

This was “an extremely unusual procedure that would seriously undermine the statistical validity and credibility of the final trial results,” Nissen stated.

The academic steering committee for the trial was convinced to report the interim results, which did not find a significantly increased risk of MI in patients taking rosiglitazone.

The final results, reported last year, also failed to show a significantly increased risk of MI but did find twice the risk of heart failure.

DeAngelis and Fontanarosa wrote, “The manufacturer of rosiglitazone exerted inappropriate influence during the conduct of a pivotal safety study of this drug … which included undertaking nonprespecified unblinding of study data, attempting to undermine the authority and responsibilities of the study steering committee, expediting publication of an unscheduled interim analysis, … having employees extensively involved in statistical analysis and preparation of the manuscript reporting the results of the trial, and reportedly failing to fully acknowledge the significant cardiovascular risk associated with this drug.”

According to Nissen, the situation with RECORD showed that the “absence of independent access to all of the data in the trial may allow physician-scientists to be manipulated by the sponsor, resulting in a manuscript that does not provide the most accurate assessment of the risks and benefits of the therapy.”

Situations like this have “become all too common,” DeAngelis and Fontanarosa wrote, and journals have implemented changes in an attempt to deal with potential conflicts.

Requiring detailed information about data management, statistical analysis, and sponsor involvement “represent important progress in achieving more transparent reporting of industry-sponsored studies,” but, they wrote, “it is virtually inevitable that other instances of incomplete or inappropriate data analysis, selective reporting, and misrepresentation of reports involving industry-sponsored research may occur.”

Requiring an independent academic statistician to perform analyses for industry-sponsored trials is another step toward dealing with this problem, they wrote.

“We recognize that this requirement … entails additional effort, time, and cost,” they wrote. “However, given concerns about the integrity and reporting of some industry-sponsored research, this independent assessment of the data and statistical analysis is essential and also provides an additional layer of institutional oversight for the integrity of the data analysis and reporting.”

Such a requirement is followed by JAMA itself and by most — but not all — of the Archives journals.

The requirement is listed as a preference for the Archives of Dermatology, and there is no such policy for the Archives of Ophthalmology or the Archives of Facial Plastic Surgery.

NEJM also does not have a requirement for independent statistical analysis of industry-sponsored trials, but has other safeguards.

According to a spokesperson for the journal, “NEJM employs four biomedical statisticians who review the appropriateness of statistical approach and analysis plan in all manuscripts before they are published. We often seek clarification, revision, and sometimes modified or additional statistical tests from study authors.”

Todd Neale is a MedPage Today staff writer.

Originally published in MedPage Today. Visit MedPageToday.com for more clinical trial news.

email

  • http://www.drmintz.com Matthew Mintz, MD

    I agree that independent analysis should be done. However, why limit this to industry sponsored studies? The assumption is that only investigators in industry sponsored studies have bias. Dr. Nissen’s published his meta-analysis with a clear bias to show a drug was dangerous, as he has done with many other drugs. His publication went from submission to publication in three weeks, which was not nearly enough time for the NEJM to carefully analyze his techniques. To separate publications in other journals re-did this analysis and came to completely different conclusions. If Dr. Nissen’s analysis was sent to and independent group, it would have never been published. Since this publication, the use of TZD medications has been cut in half. In addition, the media, including this post seems to have overlooked the fact that the final results of teh RECORD trial not only showed Avandia did not increase cardiovascular risks (other than CHF, and known side effect), but also improved diabetes control with less use of insulin and less hypoglycemia. In the recent BARI 2D study, and NIH sponosred trial, diabetics who had bypass surgery (a pretty at risk group) actually had substantially fewer heart attacks when prescribed an insulin sensitizing regimen (mostly included Avandia) as opposed to an inuslin providing regimen.
    Independent analysis is a great idea. Let’s apply the rule to all studies, and not just industry sponsored ones.

  • http://nostrums.blogspot.com James Dougherty

    I’m not opposed to independent statistical review, but I think it may be just re-arranging the deck chairs on the Titanic. Who validates the “independent” reviewer? They can be just as vulnerable to industry influence as anyone else. It’s not that hard to find out who’s doing such confidential work.

    Is creating a “who’s watching the watchers” chain any better?

  • Doc99

    I learned long ago not to jump on the bandwagon for a new drug or procedure until at least two years had passed. The pendulum of medicine swings back and forth. What is In soon becomes Out as the Parking on the Left becomes the Parking on the Right. And … then it swings back again. Count on it.

  • Frank

    YES. Including those alleged “health care reform studies” done by Harvard Law School — not economists. How amazing — the Harvard Law econometric “studies” supported the position of Mr. Obama — what a coincidence! (sarcasm)

    Those with brains know they are being conned — and do not appreciate it. There will be a reckoning.

Trending