Small Telescopes: Detectability and the Evaluation of Replication Results

被引：539

作者：

Simonsohn, Uri ^{[1
]}

机构：

[1] Univ Penn, Wharton Sch, 3730 Walnut St,500 Huntsman Hall, Philadelphia, PA 19104 USA

来源：

PSYCHOLOGICAL SCIENCE | 2015年 / 26卷 / 05期

关键词：

replication; statistical power; hypothesis testing; open materials; PSYCHOLOGICAL-RESEARCH; STATISTICAL POWER; EFFECT SIZE; TESTS; WILLINGNESS; DISPARITY; PAY;

D O I：

10.1177/0956797614567341

中图分类号：

B84 [心理学];

学科分类号：

04 ; 0402 ;

摘要：

This article introduces a new approach for evaluating replication results. It combines effect-size estimation with hypothesis testing, assessing the extent to which the replication results are consistent with an effect size big enough to have been detectable in the original study. The approach is demonstrated by examining replications of three well-known findings. Its benefits include the following: (a) differentiating unsuccessful replication attempts (i.e., studies yielding p > .05) that are too noisy from those that actively indicate the effect is undetectably different from zero, (b) protecting true findings from underpowered replications, and (c) arriving at intuitively compelling inferences in general and for the revisited replications in particular.

引用

页码：559 / 569

页数：11

共 44 条

[1] An Open, Large-Scale, Collaborative Effort to Estimate the Reproducibility of Psychological Science [J].

Alexander, Anita ;

Barnett-Cowan, Michael ;

Bartmess, Elizabeth ;

Bosco, Frank A. ;

Brandt, Mark ;

Carp, Joshua ;

Chandler, Jesse J. ;

Clay, Russ ;

Cleary, Hayley ;

Cohn, Michael ;

Costantini, Giulio ;

DeCoster, Jamie ;

Dunn, Elizabeth ;

Eggleston, Casey ;

Estel, Vivien ;

Farach, Frank J. ;

Feather, Jenelle ;

Fiedler, Susann ;

Field, James G. ;

Foster, Joshua D. ;

Frank, Michael ;

Frazier, Rebecca S. ;

Fuchs, Heather M. ;

Galak, Jeff ;

Galliani, Elisa Maria ;

Garcia, Sara ;

Giammanco, Elise M. ;

Gilbert, Elizabeth A. ;

Giner-Sorolla, Roger ;

Goellner, Lars ;

Goh, Jin X. ;

Goss, R. Justin ;

Graham, Jesse ;

Grange, James A. ;

Gray, Jeremy R. ;

Gripshover, Sarah ;

Hartshorne, Joshua ;

Hayes, Timothy B. ;

Jahn, Georg ;

Johnson, Kate ;

Johnston, William ;

Joy-Gaba, Jennifer A. ;

Lai, Calvin K. ;

Lakens, Daniel ;

Lane, Kristin ;

LeBel, Etienne P. ;

Lee, Minha ;

Lemm, Kristi ;

Mackinnon, Sean ;

May, Michael .

PERSPECTIVES ON PSYCHOLOGICAL SCIENCE, 2012, 7 (06) :657-660

[2] Registered Replication Report: Schooler and Engstler-Schooler (1990) [J].

Alogna, V. K. ;

Attaya, M. K. ;

Aucoin, P. ;

Bahnik, S. ;

Birch, S. ;

Birt, A. R. ;

Bornstein, B. H. ;

Bouwmeester, S. ;

Brandimonte, M. A. ;

Brown, C. ;

Buswell, K. ;

Carlson, C. ;

Carlson, M. ;

Chu, S. ;

Cislak, A. ;

Colarusso, M. ;

Colloff, M. F. ;

Dellapaolera, K. S. ;

Delvenne, J.-F. ;

Di Domenico, A. ;

Drummond, A. ;

Echterhoff, G. ;

Edlund, J. E. ;

Eggleston, C. M. ;

Fairfield, B. ;

Franco, G. ;

Gabbert, F. ;

Gamblin, B. W. ;

Garry, M. ;

Gentry, R. ;

Gilbert, E. A. ;

Greenberg, D. L. ;

Halberstadt, J. ;

Hall, L. ;

Hancock, P. J. B. ;

Hirsch, D. ;

Holt, G. ;

Jackson, J. C. ;

Jong, J. ;

Kehn, A. ;

Koch, C. ;

Kopietz, R. ;

Koerner, U. ;

Kunar, M. A. ;

Lai, C. K. ;

Langton, S. R. H. ;

Leite, F. P. ;

Mammarella, N. ;

Marsh, J. E. ;

McConnaughy, K. A. .

PERSPECTIVES ON PSYCHOLOGICAL SCIENCE, 2014, 9 (05) :556-578

[3]

[Anonymous], 1926, Journal of the Ministry of Agriculture, DOI DOI 10.1007/978-1-4612-4380-9_8

[4] Recommendations for Increasing Replicability in Psychology [J].

Asendorpf, Jens B. ;

Conner, Mark ;

De Fruyt, Filip ;

De Houwer, Jan ;

Denissen, Jaap J. A. ;

Fiedler, Klaus ;

Fiedler, Susann ;

Funder, David C. ;

Kliegl, Reinhold ;

Nosek, Brian A. ;

Perugini, Marco ;

Roberts, Brent W. ;

Schmitt, Manfred ;

vanAken, Marcel A. G. ;

Weber, Hannelore ;

Wicherts, Jelte M. .

EUROPEAN JOURNAL OF PERSONALITY, 2013, 27 (02) :108-119

[5] A 61-million-person experiment in social influence and political mobilization [J].

Bond, Robert M. ;

Fariss, Christopher J. ;

Jones, Jason J. ;

Kramer, Adamd. I. ;

Marlow, Cameron ;

Settle, Jaime E. ;

Fowler, James H. .

NATURE, 2012, 489 (7415) :295-298

[6] Power failure: why small sample size undermines the reliability of neuroscience [J].

Button, Katherine S. ;

Ioannidis, John P. A. ;

Mokrysz, Claire ;

Nosek, Brian A. ;

Flint, Jonathan ;

Robinson, Emma S. J. ;

Munafo, Marcus R. .

NATURE REVIEWS NEUROSCIENCE, 2013, 14 (05) :365-376

[7] STATISTICAL POWER OF ABNORMAL-SOCIAL PSYCHOLOGICAL-RESEARCH - A REVIEW [J].

COHEN, J .

JOURNAL OF ABNORMAL PSYCHOLOGY, 1962, 65 (03) :145-+

[8] THE EARTH IS ROUND (P-LESS-THAN.05) [J].

COHEN, J .

AMERICAN PSYCHOLOGIST, 1994, 49 (12) :997-1003

[9]

Cohen J., 2013, Statistical power analysis for the behavioral sciences

[10] THE DISPARITY BETWEEN WILLINGNESS TO ACCEPT AND WILLINGNESS TO PAY MEASURES OF VALUE [J].

COURSEY, DL ;

HOVIS, JL ;

SCHULZE, WD .

QUARTERLY JOURNAL OF ECONOMICS, 1987, 102 (03) :679-690

← 1 2 3 4 5 →