Evaluating replicability of laboratory experiments in economics

被引：671

作者：

Camerer, Colin F. ^{[1
]}

Dreber, Anna ^{[2
]}

Forsell, Eskil ^{[2
]}

Ho, Teck-Hua ^{[3
,4
]}

Huber, Juergen ^{[5
]}

Johannesson, Magnus ^{[2
]}

Kirchler, Michael ^{[5
,6
]}

Almenberg, Johan ^{[7
]}

Altmejd, Adam ^{[2
]}

Chan, Taizan ^{[8
]}

Heikensten, Emma ^{[2
]}

Holzmeister, Felix ^{[5
]}

Imai, Taisuke ^{[1
]}

Isaksson, Siri ^{[2
]}

Nave, Gideon ^{[1
]}

Pfeiffer, Thomas ^{[9
,10
]}

Razen, Michael ^{[5
]}

Wu, Hang ^{[4
]}

机构：

[1] CALTECH, Div Humanities & Social Sci, 1200 East Calif Blvd,MC 228-77, Pasadena, CA 91125 USA

[2] Stockholm Sch Econ, Dept Econ, Box 6501, SE-11383 Stockholm, Sweden

[3] Univ Calif Berkeley, Haas Sch Business, Berkeley, CA 94720 USA

[4] Natl Univ Singapore, NUS Business Sch, Singapore 119245, Singapore

[5] Univ Innsbruck, Dept Banking & Finance, Univ Str 15, A-6020 Innsbruck, Austria

[6] Univ Gothenburg, Dept Econ, Ctr Finance, SE-40530 Gothenburg, Sweden

[7] Sveriges Riksbank, SE-10337 Stockholm, Sweden

[8] Natl Univ Singapore, Off Deputy President Res & Technol, Singapore 119077, Singapore

[9] New Zealand Inst Adv Study, North Shore Mail Ctr, Private Bag 102904, Auckland 0745, New Zealand

[10] Inst Adv Study, Wissensch Kolleg Berlin, D-14193 Berlin, Germany

来源：

SCIENCE | 2016年 / 351卷 / 6280期

基金：

奥地利科学基金会;

关键词：

REPLICATION; REPRODUCIBILITY; EXCHANGE; POLICY; CANCER;

D O I：

10.1126/science.aaf0918

中图分类号：

O [数理科学和化学]; P [天文学、地球科学]; Q [生物科学]; N [自然科学总论];

学科分类号：

07 ; 0710 ; 09 ;

摘要：

The replicability of some scientific findings has recently been called into question. To contribute data about replicability in economics, we replicated 18 studies published in the American Economic Review and the Quarterly Journal of Economics between 2011 and 2014. All of these replications followed predefined analysis plans that weremade publicly available beforehand, and they all have a statistical power of at least 90% to detect the original effect size at the 5% significance level. We found a significant effect in the same direction as in the original study for 11 replications (61%); on average, the replicated effect size is 66% of the original. The replicability rate varies between 67% and 78% for four additional replicability indicators, including a prediction market measure of peer beliefs.

引用

页码：1433 / 1436

页数：4

共 48 条

[1] Reference Points and Effort Provision [J].

Abeler, Johannes ;

Falk, Armin ;

Goette, Lorenz ;

Huffman, David .

AMERICAN ECONOMIC REVIEW, 2011, 101 (02) :470-492

[2] Imperfect Public Monitoring with Costly Punishment: An Experimental Study [J].

Ambrus, Attila ;

Greiner, Ben .

AMERICAN ECONOMIC REVIEW, 2012, 102 (07) :3317-3332

[3]

[Anonymous], 2015, SCIENCE, DOI [DOI 10.1126/science.aac4716, DOI 10.1126/SCIENCE.AAC4716]

[4] Economics - The promise of prediction markets [J].

Arrow, Kenneth J. ;

Forsythe, Robert ;

Gorham, Michael ;

Hahn, Robert ;

Hanson, Robin ;

Ledyard, John O. ;

Levmore, Saul ;

Litan, Robert ;

Milgrom, Paul ;

Nelson, Forrest D. ;

Neumann, George R. ;

Ottaviani, Marco ;

Schelling, Thomas C. ;

Shiller, Robert J. ;

Smith, Vernon L. ;

Snowberg, Erik ;

Sunstein, Cass R. ;

Tetlock, Paul C. ;

Tetlock, Philip E. ;

Varian, Hal R. ;

Wolfers, Justin ;

Zitzewitz, Eric .

SCIENCE, 2008, 320 (5878) :877-878

[5] Screening, Competition, and Job Design: Economic Origins of Good Jobs [J].

Bartling, Bjoern ;

Fehr, Ernst ;

Schmidt, Klaus M. .

AMERICAN ECONOMIC REVIEW, 2012, 102 (02) :834-864

[6] Raise standards for preclinical cancer research [J].

Begley, C. Glenn ;

Ellis, Lee M. .

NATURE, 2012, 483 (7391) :531-533

[7] Replication effort provokes praise-and 'bullying' charges [J].

Bohannon, John .

SCIENCE, 2014, 344 (6186) :788-789

[8] Star Wars: The Empirics Strike Back [J].

Brodeur, Abel ;

Le, Mathias ;

Sangnier, Marc ;

Zylberberg, Yanos .

AMERICAN ECONOMIC JOURNAL-APPLIED ECONOMICS, 2016, 8 (01) :1-32

[9] Power failure: why small sample size undermines the reliability of neuroscience [J].

Button, Katherine S. ;

Ioannidis, John P. A. ;

Mokrysz, Claire ;

Nosek, Brian A. ;

Flint, Jonathan ;

Robinson, Emma S. J. ;

Munafo, Marcus R. .

NATURE REVIEWS NEUROSCIENCE, 2013, 14 (05) :365-376

[10] Psychology's Bold Initiative [J].

Carpenter, Siri .

SCIENCE, 2012, 335 (6076) :1558-1561

← 1 2 3 4 5 →