CLC Submits Comments on NIH Evaluation of Peer Review System
March 17, 2008
Delivered by email to PeerReviewRFI@mail.nih.gov
Dear Dr. Zerhouni:
The undersigned cancer patient, provider, and research organizations are pleased to submit comments regarding the National Institutes of Health (NIH) peer review self study. We commend the agency for setting goals of improving the efficiency of the peer review system and enhancing the emphasis on funding new research proposals instead of resubmitted applications. We propose some additional reforms that are consistent with these goals and we also offer cautions regarding some of the reform recommendations.
Encourage Participation of Patient Advocates and Clinical Researchers in Peer Review
The peer review report: 1) emphasizes the need to fund research that has a significant impact and is new and transformative and 2) recommends a number of reforms that are intended to improve the funding success of such proposals. One proposed reform is a two-stage review process; during the first part of the two-step process, the research application would be assessed for its potential impact and its likelihood of success and during the second part of the process the technical merits of the application would be reviewed.
Although we have no objections to this reform, we recommend two more fundamental steps be taken prior to implementing a two-stage review process. First, clinical researchers should have central responsibility for the review of clinical research proposals, as they are well equipped to evaluate the potential impact of such proposals as well as their potential for success. The report acknowledges that clinical research proposals have a lower success rate than non-clinical applications. However, the report does not state that clinical researchers should have a key role in review of these applications. We strongly recommend that this standard for review be established and followed.
Second, we strongly recommend that patient advocates be included in review of clinical research proposals. The report states: “Some argue that the presence of ‘non-scientists’ erodes peer review, since a non-scientist is not a ‘peer.’ Others have argued that patients or their advocates have unique insight into certain aspects of clinical research, including the feasibility of proposed work -- particularly with regard to recruitment issues and human subjects protections.” There is strong evidence from other peer review programs that patient advocates strengthen rather than erode peer review, and we urge NIH to include them in study sections evaluating clinical research proposals.
Patient advocates provide an invaluable perspective regarding the viability and importance of clinical research proposals. They are often exceptionally well-informed about the state of research in a particular field and about existing treatment options; they can apply that knowledge to give advice about the potential impact of a clinical study. Moreover, they can advise regarding the likelihood that patients will enroll in proposed clinical studies. Patient advocates can provide special guidance about the viability of clinical research efforts.
We do not suggest that researchers be named to study sections without a consideration of their qualifications as peer reviewers. Neither would we suggest that patient advocates be appointed without an evaluation of the skills, expertise, and experience that would inform their work as peer reviewers. A significant number of patient advocates have served as peer reviewers in the Department of Defense Congressionally Directed Medical Research Program and other federal agencies and in private research foundations. In addition, many advocacy organizations conduct rigorous training programs to enhance the skills and knowledge of advocates to serve as peer reviewers. As a result, we believe that NIH will find a significant cohort of patient advocates with appropriate skills to serve as peer reviewers. We strongly recommend that NIH act immediately to include patient advocates in clinical research review on a routine and regular basis.
Review of Clinical Research
Although the report acknowledges that clinical research applications have lower success rates than non-clinical research proposals, it conveys no urgency about addressing this situation. Instead, the report recommends further study to understand the application submission trends of clinical researchers.
We recommend immediate action. In addition to involving clinical researchers and patient advocates in the review of clinical research applications (as outlined above), we recommend that these applications be considered by special panels at the Institute and Center (IC) level rather than by study sections convened by the Center for Scientific Review. In addition, we propose that NIH consider a more efficient means for addressing human subjects concerns, which it has identified as a special obstacle to timely and efficient review of clinical research proposals.
As advocates who seek the rapid translation of basic research findings into new therapies, we support greater efficiency and quality in the review of clinical research applications.
Elimination of Resubmissions
We applaud NIH for offering aggressive recommendations to address the current system in which investigators are often not funded until the third round of review. The report proposes the elimination of the category of “resubmitted” grants, establishes a process for feedback and correction of problems in grant applications that might forestall the need for resubmission, and provides investigators clear information regarding their grants in the form of a “Not Recommended for Resubmission” recommendation.
As clinical researchers and patient advocates who want to see NIH fund new and transformative research that will improve treatment of cancer and other serious diseases, we support the effort to improve the peer review system so that it rewards such applications. However, we are concerned that there may be serious dislocations as these standards of review are implemented. We urge NIH to consider the manner in which it implements these processes for review.
We believe that it is the stagnation in NIH funding -- and not simply the current standards and processes for peer review -- that influences grant funding decisions. In the face of restraints on funding, peer reviewers are reportedly becoming more conservative and tend to reward “safer” applications. In turn, applicants are adjusting to these standards by making their proposals more modest in scope and ambition. We do not applaud this trend, but neither do we think that the transition to a system that more significantly rewards risk taking will be easy.
We appreciate the opportunity to comment on proposals to improve peer review, and we are available to provide additional advice regarding the role of clinical researchers and patient advocates in the review of clinical research.
Cancer Leadership Council
American Cancer Society Cancer
© 2001-2002 Cancer Leadership Council. All rights reserved.