Evaluating replicability of laboratory experiments in economics

被引：671

作者：

Camerer, Colin F. ^{[1
]}

Dreber, Anna ^{[2
]}

Forsell, Eskil ^{[2
]}

Ho, Teck-Hua ^{[3
,4
]}

Huber, Juergen ^{[5
]}

Johannesson, Magnus ^{[2
]}

Kirchler, Michael ^{[5
,6
]}

Almenberg, Johan ^{[7
]}

Altmejd, Adam ^{[2
]}

Chan, Taizan ^{[8
]}

Heikensten, Emma ^{[2
]}

Holzmeister, Felix ^{[5
]}

Imai, Taisuke ^{[1
]}

Isaksson, Siri ^{[2
]}

Nave, Gideon ^{[1
]}

Pfeiffer, Thomas ^{[9
,10
]}

Razen, Michael ^{[5
]}

Wu, Hang ^{[4
]}

机构：

[1] CALTECH, Div Humanities & Social Sci, 1200 East Calif Blvd,MC 228-77, Pasadena, CA 91125 USA

[2] Stockholm Sch Econ, Dept Econ, Box 6501, SE-11383 Stockholm, Sweden

[3] Univ Calif Berkeley, Haas Sch Business, Berkeley, CA 94720 USA

[4] Natl Univ Singapore, NUS Business Sch, Singapore 119245, Singapore

[5] Univ Innsbruck, Dept Banking & Finance, Univ Str 15, A-6020 Innsbruck, Austria

[6] Univ Gothenburg, Dept Econ, Ctr Finance, SE-40530 Gothenburg, Sweden

[7] Sveriges Riksbank, SE-10337 Stockholm, Sweden

[8] Natl Univ Singapore, Off Deputy President Res & Technol, Singapore 119077, Singapore

[9] New Zealand Inst Adv Study, North Shore Mail Ctr, Private Bag 102904, Auckland 0745, New Zealand

[10] Inst Adv Study, Wissensch Kolleg Berlin, D-14193 Berlin, Germany

来源：

SCIENCE | 2016年 / 351卷 / 6280期

基金：

奥地利科学基金会;

关键词：

REPLICATION; REPRODUCIBILITY; EXCHANGE; POLICY; CANCER;

D O I：

10.1126/science.aaf0918

中图分类号：

O [数理科学和化学]; P [天文学、地球科学]; Q [生物科学]; N [自然科学总论];

学科分类号：

07 ; 0710 ; 09 ;

摘要：

The replicability of some scientific findings has recently been called into question. To contribute data about replicability in economics, we replicated 18 studies published in the American Economic Review and the Quarterly Journal of Economics between 2011 and 2014. All of these replications followed predefined analysis plans that weremade publicly available beforehand, and they all have a statistical power of at least 90% to detect the original effect size at the 5% significance level. We found a significant effect in the same direction as in the original study for 11 replications (61%); on average, the replicated effect size is 66% of the original. The replicability rate varies between 67% and 78% for four additional replicability indicators, including a prediction market measure of peer beliefs.

引用

页码：1433 / 1436

页数：4

共 48 条

[41] Promoting an open research culture [J].

Nosek, B. A. ;

Alter, G. ;

Banks, G. C. ;

Borsboom, D. ;

Bowman, S. D. ;

Breckler, S. J. ;

Buck, S. ;

Chambers, C. D. ;

Chin, G. ;

Christensen, G. ;

Contestabile, M. ;

Dafoe, A. ;

Eich, E. ;

Freese, J. ;

Glennerster, R. ;

Goroff, D. ;

Green, D. P. ;

Hesse, B. ;

Humphreys, M. ;

Ishiyama, J. ;

Karlan, D. ;

Kraut, A. ;

Lupia, A. ;

Mabry, P. ;

Madon, T. A. ;

Malhotra, N. ;

Mayo-Wilson, E. ;

McNutt, M. ;

Miguel, E. ;

Paluck, E. Levy ;

Simonsohn, U. ;

Soderberg, C. ;

Spellman, B. A. ;

Turitto, J. ;

VandenBos, G. ;

Vazire, S. ;

Wagenmakers, E. J. ;

Wilson, R. ;

Yarkoni, T. .

SCIENCE, 2015, 348 (6242) :1422-1425

[42] Believe it or not: how much can we rely on published data on potential drug targets? [J].

Prinz, Florian ;

Schlange, Thomas ;

Asadullah, Khusru .

NATURE REVIEWS DRUG DISCOVERY, 2011, 10 (09) :712-U81

[43] Failing the Future: Three Unsuccessful Attempts to Replicate Bem's 'Retroactive Facilitation of Recall' Effect [J].

Ritchie, Stuart J. ;

Wiseman, Richard ;

French, Christopher C. .

PLOS ONE, 2012, 7 (03)

[44]

Roth A.E., 1994, Empirical Economics, V19, P279, DOI DOI 10.1007/BF01175875

[45] False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant [J].

Simmons, Joseph P. ;

Nelson, Leif D. ;

Simonsohn, Uri .

PSYCHOLOGICAL SCIENCE, 2011, 22 (11) :1359-1366

[46] Small Telescopes: Detectability and the Evaluation of Replication Results [J].

Simonsohn, Uri .

PSYCHOLOGICAL SCIENCE, 2015, 26 (05) :559-569

[47] Bayesian Tests to Quantify the Result of a Replication Attempt [J].

Verhagen, Josine ;

Wagenmakers, Eric-Jan .

JOURNAL OF EXPERIMENTAL PSYCHOLOGY-GENERAL, 2014, 143 (04) :1457-1475

[48]

Wolfers J., 2006, 12200 NAT BUR EC RES

← 1 2 3 4 5 →