Bioscience Briefing
For info about biomedical funding, animal research and other bioscience issues, click here.
Science Policy Announcements
For the latest news from the APS Science Policy click here.
Resource Book Now Available
Click here for more info on The APS Resource Book for the Design of Animal Exercise Protocols.
NIH Announces Changes to Peer Review System
On June 6, 2008 the National Institutes of Health (NIH) announced a series of changes intended to enhance the peer review system that will be implemented in the coming months. The initiatives are the culmination of a year-long effort on the part of NIH to update a system that is struggling to deal with flat budgets as the number of applications soars.
A task force led by Drs. Lawrence Tabak and Keith Yamamoto released an 88-page report in February. The NIH evaluated it internally and asked for feedback from the scientific community. As a result, some of the more controversial proposals were dropped including establishing a “not recommended for resubmission” designation, treating all applications as new, and instituting a minimum percent effort requirement.
The changes are organized around four main priorities: engaging the best reviewers, improving the quality and transparency of reviews, ensuring fair and balanced reviews across scientific fields and career stages, and developing a permanent process for continuous review of peer review. The changes associated with each of the four priorities are summarized below. Some of the major modifications include shortening R01 grant applications to 12 pages, providing administrative supplements for reviewers who serve 18 full study section meetings as chartered members, and scoring applications on a 7 point scale.
Priority 1. Engage the best reviewers
In order to attract and retain the best reviewers, the NIH plans to increase the flexibility of reviewer service by allowing the 12 session reviewer commitment to be spread over 4-6 years. There will also be an expansion of flexible submission deadlines for reviewers and pilots for new forms of electronic review.
Certain awards including Merit/Javits and Pioneer awards will now include a “service expectation.” PIs with three or more R01 equivalents or type 2 renewals that exceed $500,000 in direct costs will now be expected to serve as peer reviewers.
To compensate the time and effort spent on review, reviewers who serve a minimum of 18 full study section meetings as chartered members will be eligible to apply for an administrative supplement of up to $250,000. Those individuals may also request consideration for Merit/Javits awards on a competitive basis.
Finally, training and mentoring will be provided to all study section chairs, reviewers and SROs using an NIH-wide curriculum based on best practices, augmented by IC and study section specific additions.
Priority 2. Improve the quality and transparency of reviews
The rating system will be modified to focus on specific review criteria, placing less emphasis on methodology, and more on scientific impact. Individual scores will be given for each of five review criteria and assigned on a scale of 1–7, instead of the current 41 point scale. The five criteria that will be scored are impact, innovation/originality, investigator, project plan/feasibility, and environment. Following the initial scoring, applications will be grouped and ranked within relevant categories. Any applications that are streamlined will be given the averaged scores on all five criteria.
To go along with the revisions in the scoring system, the summary statement will be structured to align with the review criteria. An optional field for “mentoring advice” will be provided and could include a recommendation not to resubmit the application unless fundamentally revised as a new proposal. NIH plans to develop appropriate tools, guidance, and training for reviewers in order to establish best practices for generating summary statements.
Under the new system, the length of R01 applications will be reduced to 12 pages, with other mechanisms scaled appropriately. Appendices will be allowed for specific information such as elements of a clinical trial.
Priority 3. Ensure balanced and fair reviews across scientific fields and career stages
One of NIH’s goals is to continue to support and develop policies to fund a minimum number of early stage investigators (ESIs) and new investigators. To that end, there are plans to cluster review, discussion, scoring and ranking of ESIs within study sections, and pilot percentiling ESIs across all study sections.
In an effort to make sure that experienced reviewers get fair evaluations, there will be equal emphasis on retrospective assessment of accomplishments and a prospective assessment of what is being proposed.
Clinical research applications will also be clustered for review, discussion, scoring and ranking within a study section.
To encourage risk taking by applicants, the review process that was initiated for the Pioneer, EUREKA and New Innovator awards will be expanded, and the Transformative Research portfolio will be grown to reach ~1% of R01-like awards.
Finally, based on analysis of success rates as a function of initial scores, NIH will work to reduce the need for resubmissions by carefully rebalancing success rates among A0, A1 and A2 submissions.
Priority 4. Develop a permanent process for continuous review of peer review
Recognizing the need to carry out ongoing review of the peer review system, NIH will continue to pilot and evaluate new models of review (i.e. editorial board models, use of prebuttals), pilot and evaluate difference methods for ranking relative merit of applications, pilot and evaluate high bandwidth electronic review and develop metrics for monitoring performance of peer review.
The next step in this process will be the formation of an ad hoc peer review task force that will develop plans and oversee implementation. To learn more about the implementation process and the changes under consideration, go to http://enhancing-peer-review.nih.gov.
Back to NIH Policies