EVALUATION OF PROPOSED AND EXISTING ACCELERATED RESEARCH PROGRAMS BY THE OFFICE-OF-NAVAL-RESEARCH

A process for evaluating proposed and existing accelerated research programs is described. The key component of the process is the use of mixed-level review panels, consisting of bench-level researchers, research managers, transitioning experts, technology experts, and US Navy requirements experts, to review multiple programs (nominally) in technical areas of similar science. The panel members score specific research quality and mission relevance rating factors as well as an overall program quality rating factor. As evidenced by multiple regression analyses, the research quality and mission relevance factors that the reviewers rated have been their dominant considerations in determining the overall program-quality rating-factor score. This implies that rating factors that the reviewers felt were important were not omitted. The operational components of the review process are described in detail, and results of experiments in which program presentation, evaluation, and discussion times were varied are presented. Application experience of the process to the US Office of Naval Research program evaluation is presented.