On May 7, the National Academies of Science, Engineering, and Medicine (NASEM) released findings and recommendations from the congressionally mandated study, “Reproducibility and Replicability in Science.”

Guided by the Committee on Reproducibility and Replicability and chaired by Harvey V. Fineberg, MD, PhD, president of the Gordon and Betty Moore Foundation and former Institute of Medicine president, the 18-month effort is a trans-disciplinary examination of factors contributing to the inability to reproduce or replicate published work and existing best practices to mitigate these issues.

An immediate issue tackled by the committee was confusion about use of the terms “reproducibility” and “replicability.” While widely regarded as interchangeable, each in fact refers to a distinct concept. “Reproducibility” refers to computational reproducibility or the ability to obtain consistent results using the same inputs, steps, methods/code, and conditions. “Replicability” refers to the ability to independently obtain consistent results. These definitions echo those established by FASEB in its 2016 report, “Enhancing Research Reproducibility.”

The committee also determined that, while improvements are needed to enhance reproducibility and replicability across scientific disciplines, there are no indications that the research community is complacent in addressing these concerns, nor is there a widespread crisis. Rather, these issues are very nuanced and multi-factorial. As noted in the findings, the inability to reproduce an experiment is not always a negative outcome. In fact, the inability to replicate an experiment may help identify factors that could further scientific progress.

Recognizing that public perception of science matters, the committee offered recommendations for researchers to communicate the nuances of research findings and resist the urge to overstate outcomes or downstream applications. These recommendations echoed discussions during the 2017 workshop on “Responsible Communication of Basic Biomedical Research: Enhancing Awareness and Avoiding Hype,” co-hosted by FASEB and the National Institute of General Medical Sciences.

A symposium on the topic of reproducibility and replicability in science will be hosted by NASEM on September 24, 2019.