Episode Details

Last year, Pew Research reported that only 29% of Americans now are willing to say they have a “great deal of confidence” in medical scientists to act in the best interests of the public. That represents an 11% decline since 2020. This dramatic drop is both significant, given the historic importance of medical research in shaping public opinion, and understandable, given a growing crisis in the reliability of scientific research overall.  

A year ago, in a Breakpoint commentary, we described this crisis. For example, according to an analysis by University of California behavioral economists, the least reliable scientific studies are most likely to be cited by other scientists. After a review of 20,000 published papers, these researchers suggested in an article for the journal Science, that doubtful findings are cited more often because they’re “interesting.” 

And now, the problem has led some scientists to “moonlight” as detectives, combing through the scientific literature to sniff out fraud, negligence, and mistakes. A recent article in The Wall Street Journal described one such sleuthing trio. Joe Simmons, Leif Nelson, and Uri Simonsohn run a website called Data Colada, which is dedicated to “debunking published studies built on faulty or fraudulent data.” According to the article, these scientists are able to recognize suspicious patterns in scientific papers, such as cherry-picked data, small sample sizes, bad math, or just results that make no sense.  

In a sense, these moonlighters are doing the kind of work that scientists should be doing as a normal part of their work. However, the scientific enterprise is plagued by what has been called a “replication crisis.” In essence, findings are too often published without anyone confirming the results with other experiments. This became common knowledge in 2016 when the journal Nature reported that “more than 70% of researchers have tried and failed to reproduce another scientist’s experiments, and more than half have failed to reproduce their own experiments.” 

Thanks in large part to the efforts of sleuths like Data Colada, “[a]t least 5,500 faulty papers were retracted in 2022, compared with (only) 119 in 2002.” All the debunking has led to embarrassing resignations, including the former president of Stanford University, as well as “upended careers and retaliatory lawsuits.” And this is probably just the beginning. According to The Wall Street Journal report, of the nearly 800 papers one researcher reported in the last decade, “only a third had been corrected or retracted five years later.” 

Of course, human fallenness is behind this mess. That may sound like an oversimplification, but it’s significant considering the myth of the objective scientist always following wherever the evidence leads. In addition to faulty and fraudulent results being more “interesting,” there are material incentives to fudge research. Pumping out papers “can yield jobs, grants, spea