Friday, October 10, 2008

Do Top Journals Have a Tendency to Publish Headline-Grabbing but Wrong Research?

This article in the magazine The Economist talks about research by Ioannidis that suggests that a large part of highly cited research published in leading journals is refuted only a few years later:
Dr Ioannidis based his earlier argument about incorrect research partly on a study of 49 papers in leading journals that had been cited by more than 1,000 other scientists. They were, in other words, well-regarded research. But he found that, within only a few years, almost a third of the papers had been refuted by other studies. ...
In a more recent article he and his coauthors argue that research published in trendy fields and top journals appears to be more prone to error:

... Dr Ioannidis and his colleagues argue that the reputations of the journals are pumped up by an artificial scarcity of the kind that keeps diamonds expensive. And such a scarcity, they suggest, can make it more likely that the leading journals will publish dramatic, but what may ultimately turn out to be incorrect, research. ...

The group’s more general argument is that scientific research is so difficult—the sample sizes must be big and the analysis rigorous—that most research may end up being wrong. And the “hotter” the field, the greater the competition is and the more likely it is that published research in top journals could be wrong.

No comments: