Conceptualizing and evaluating the replication of research results

Many recent discussions have focused on the role of replication in psychological science. In this article, we examine three key issues in evaluating the conclusions that follow from results of studies at least partly aimed at replicating previous results: the evaluation and status of exact versus conceptual replications, the statistical evaluation of replications, and the robustness of research findings to potential existing or future "non-replications." In the first section of the article, we discuss the sources of ambiguity in evaluating failures to replicate in exact as well as conceptual replications. In addressing these ambiguities, we emphasize the key role of psychometric invariance of the independent and dependent variables in evaluations of replications. In the second section of the article, we use a meta-analytic framework to discuss the statistical status of replication attempts. We emphasize meta-analytic tools that have been used too sparingly, especially in evaluation of sets of studies within a single article or focused program of research. In the final section of the article, we extend many of these meta-analytic tools to the evaluation of the robustness of a body of research to potential existing or future failures to replicate previous statistically significant results. (C) 2015 Elsevier Inc. All rights reserved.