March 10, 2014

SCIENCE: When Studies Are Wrong.

All scientific results are, of course, subject to revision and refutation by later experiments. The problem comes when these replications don’t occur and the information keeps spreading unchecked.

Dr. Ioannidis’s analysis took into account several factors — things like noisy data, a small sample size or relatively lenient standards for deciding if a finding is statistically significant. His model could be applied to any area of science that met his criteria. But most attention to the reproducibility problem has been in the life sciences, particularly in medical laboratory research and epidemiology. Based on the number of papers in major journals, Dr. Ioannidis estimates that the field accounts for some 50 percent of published research.

Another area of concern has been the social sciences, including psychology, which make up about 25 percent of publications. Together that constitutes most of scientific research. The remaining slice is physical science — everything from geology and climatology to cosmology and particle physics. These fields have not received the same kind of scrutiny as the others. Is that because they are less prone to the problems Dr. Ioannides described?

I predict that the physical sciences will have their share of fraud too. As Thomas Ray has said, every successful system accumulates parasites, and science has been successful long enough to pick up a substantial load.