NIH Announces Plans to Address Rigor and Reproducibility in Research

On June 9, 2015 the NIH announced plans to revise grant application instructions and review criteria to enhance the reproducibility of research findings through “increased scientific rigor and transparency.”

The revised instructions will require applicants to address four areas:

  • Scientific premise of proposed research. NIH will expect applicants to describe the strengths and weaknesses of prior research being cited as crucial to support the application. Description of strengths and weaknesses might include attention to the rigor of experimental design and consideration of relevant biological variables in the cited work.
  • Rigorous experimental design. NIH expects applicants to describe how they will achieve robust and unbiased results in the proposed methods and experimental design. Full transparency in reporting experimental details is also required.
  • Consideration of relevant biological variables, including sex, in vertebrate animal and human studies. Research proposals will need to detail how relevant biological variables will be considered in experimental design, analysis and interpretation. Justification will be required for research that does not propose to consider sex as a biological variable. Additional details on this topic were published in a second notice.
  • Authentication of key biological and/or chemical resources. NIH will expect key resources, including cell lines, antibodies, specialty chemicals and other biologics, to be regularly authenticated to ensure their identity and validity.

Description of scientific premise, experimental design, and consideration of biological variables will be included in the Research Strategy section of applications. Page limits remain unchanged. Reviewers will evaluate scientific premise as part of the significance criteria, and experimental design and consideration of variables as part of the approach criteria. All three of these areas will factor in to assessment of overall impact score. Authentication of resources will be included as a new attachment, which reviewers will comment on, but not consider in scoring overall impact.

NIH is currently seeking approval for incorporation of these guidelines from the Office of Management and Budget. Revised application instructions are expected to be available in the fall of 2015, for application submission for the January 25, 2016 due date. Next year grant reviewers will be trained to evaluate these new criteria.

Proposal of the new application and review criteria is one approach that the NIH is taking to address problems with reproducibility of research results. For a broad overview of the issue, including presentations from the Science Policy Committee’s 2015 Symposium on Reproducibility.