- Published research findings are sometimes refuted by subsequent evidence, with ensuing confusion and disappointment. Refutation and controversy is seen across the range of research designs, from clinical trials and traditional epidemiological studies [1–3] to the most modern molecular research [4,5]. There is increasing concern that in modern research, false findings may be the majority or even the vast majority of published research claims [6–8]. However, this should not be surprising. It can be proven that most claimed research findings are false. Here I will examine the key factors that influence this problem and some corollaries thereof.
This paper gets cited a lot in the pop-science world. It's good, very interesting, but also frequently misinterpreted and thrown around obscure internet threads as evidence that (vaccines aren't safe|big pharma is bad|academia is a waste of taxpayer money|etc). FWIW, this has been the best layman's explanation of this paper that I've come across: TL;DR: Small studies are more prone to error. Picking any one study at random means you'll probably be wrong. Weighting them in context of that field of research can help. We need more large studies.
I think one of the bigger takeaways is how research and funding is geared towards flashy results, or at least, "results" in the sense that confirming a null hypothesis isn't exciting. Larger data sets, and replication of existing studies would be fantastic. And, I'm not sure if this is the case or not, but people seeking out to confirm their own opinion and obscuring p-values to meet it. At least, that seems like a possibility.