Evaluating technology programs: tools and methods

This article reviews the analytical tools, methods, and designs being used to evaluate public programs intended to stimulate technological advance. The review is organized around broad policy categories rather than particular types of policy intervention, because methods used are rarely chosen independent of context. The categories addressed include publicly-supported research conducted in universities and public sector research organizations; evaluations of linkages, especially those programs seeking to promote academic-industrial and public-private partnerships; and the evaluation of diffusion and industrial extension programs. The internal evaluation procedures of science such as peer review and bibliometrics are not covered, nor are methods used to evaluate projects and individuals ex ante. Among the conclusion is the observation that evaluation work has had less of an impact in the literature than it deserves, in part because much of the most detailed and valuable work is not easily obtainable. A second conclusion is that program evaluations and performance reviews, which have distinctive objectives, measures, and tools, are becoming entangled, with the lines between the two becoming blurred. Finally, new approaches to measuring the payoffs from research that focus on linkages between knowledge producers and users, and on the characteristics of research networks, appear promising as the limitations of the production function and related methods have become apparent. (C) 2000 Elsevier Science B.V. All rights reserved.