Response to Comment on "Estimating the reproducibility of psychological science"

被引：103

作者：

Anderson, Christopher J. ^{[1
]}

Bahnik, Stepan ^{[2
]}

Barnett-Cowan, Michael ^{[3
]}

Bosco, Frank A. ^{[4
]}

Chandler, Jesse ^{[5
,6
]}

Chartier, Christopher R. ^{[7
]}

Cheung, Felix ^{[8
]}

Christopherson, Cody D. ^{[9
]}

Cordes, Andreas ^{[10
]}

Cremata, Edward J. ^{[11
]}

Della Penna, Nicolas ^{[12
]}

Estel, Vivien ^{[13
]}

Fedor, Anna ^{[14
]}

Fitneva, Stanka A. ^{[15
]}

Frank, Michael C. ^{[16
]}

Grange, James A. ^{[17
]}

Hartshorne, Joshua K. ^{[18
]}

Hasselman, Fred ^{[19
]}

Henninger, Felix ^{[20
]}

van der Hulst, Marije ^{[21
]}

Jonas, Kai J. ^{[22
]}

Lai, Calvin K. ^{[23
]}

Levitan, Carmel A. ^{[24
]}

Miller, Jeremy K. ^{[25
]}

Moore, Katherine S. ^{[26
]}

Meixner, Johannes M. ^{[27
]}

Munafo, Marcus R. ^{[28
]}

Neijenhuijs, Koen I. ^{[29
]}

Nilsonne, Gustav ^{[30
]}

Nosek, Brian A. ^{[31
,32
]}

Plessow, Franziska ^{[33
]}

Prenoveau, Jason M. ^{[34
]}

Ricker, Ashley A. ^{[35
]}

Schmidt, Kathleen ^{[36
]}

Spies, Jeffrey R. ^{[31
,32
]}

Stieger, Stefan ^{[37
]}

Strohminger, Nina ^{[38
]}

Sullivan, Gavin B. ^{[39
]}

van Aert, Robbie C. M. ^{[40
]}

van Assen, Marcel A. L. M. ^{[40
,41
]}

Vanpaemel, Wolf ^{[42
]}

Vianello, Michelangelo ^{[43
]}

Voracek, Martin ^{[44
]}

Zuni, Kellylynn ^{[45
]}

机构：

[1] Russell Sage Coll, Troy, NY USA

[2] Univ Wurzburg, D-97070 Wurzburg, Germany

[3] Univ Waterloo, Waterloo, ON N2L 3G1, Canada

[4] Virginia Commonwealth Univ, Richmond, VA USA

[5] Univ Michigan, Ann Arbor, MI 48104 USA

[6] Mathemat Policy Res, Washington, DC USA

[7] Ashland Univ, Ashland, OH USA

[8] Michigan State Univ, E Lansing, MI 48824 USA

[9] Southern Oregon Univ, Ashland, OR USA

[10] Univ Gottingen, Inst Psychol, D-37073 Gottingen, Germany

[11] Univ So Calif, Los Angeles, CA USA

[12] Australian Natl Univ, Canberra, ACT, Australia

[13] Tech Univ Carolo Wilhelmina Braunschweig, D-38106 Braunschweig, Germany

[14] Parmenides Stiftung, Munich, Germany

[15] Queens Univ, Kingston, ON, Canada

[16] Stanford Univ, Stanford, CA 94305 USA

[17] Keele Univ, Keele ST5 5BG, Staffs, England

[18] Boston Coll, Chestnut Hill, MA 02167 USA

[19] Radboud Univ Nijmegen, NL-6525 ED Nijmegen, Netherlands

[20] Univ Koblenz Landau, Landau, Germany

[21] Erasmus MC, Rotterdam, Netherlands

[22] Univ Amsterdam, Amsterdam, Netherlands

[23] Harvard Univ, Cambridge, MA 02138 USA

[24] Occidental Coll, Los Angeles, CA 90041 USA

[25] Willamette Univ, Salem, OR 97301 USA

[26] Arcadia Univ, Glenside, PA USA

[27] Univ Potsdam, Potsdam, Germany

[28] Univ Bristol, Bristol, Avon, England

[29] Vrije Univ Amsterdam, Amsterdam, Netherlands

[30] Stockholm Univ, Karolinska Inst, S-10691 Stockholm, Sweden

[31] Ctr Open Sci, Charlottesville, VA USA

[32] Univ Virginia, Charlottesville, VA USA

[33] Harvard Univ, Sch Med, Boston, MA USA

[34] Loyola Univ, Baltimore, MD USA

[35] Univ Calif Riverside, Riverside, CA 92521 USA

[36] Wesleyan Univ, Middletown, CT USA

[37] Univ Konstanz, Constance, Germany

[38] Yale Univ, New Haven, CT USA

[39] Coventry Univ, Coventry, W Midlands, England

[40] Tilburg Univ, NL-5000 LE Tilburg, Netherlands

[41] Univ Utrecht, Utrecht, Netherlands

[42] Univ Leuven, Leuven, Belgium

[43] Univ Padua, Padua, Italy

[44] Univ Vienna, Vienna, Austria

[45] Adams State Univ, Alamosa, CO USA

来源：

SCIENCE | 2016年 / 351卷 / 6277期

关键词：

REPLICATION;

D O I：

10.1126/science.aad9163

中图分类号：

O [数理科学和化学]; P [天文学、地球科学]; Q [生物科学]; N [自然科学总论];

学科分类号：

07 ; 0710 ; 09 ;

摘要：

Gilbert et al. conclude that evidence from the Open Science Collaboration's Reproducibility Project: Psychology indicates high reproducibility, given the study methodology. Their very optimistic assessment is limited by statistical misconceptions and by causal inferences from selectively interpreted, correlational data. Using the Reproducibility Project: Psychology data, both optimistic and pessimistic conclusions about reproducibility are possible, and neither are yet warranted.

引用

页数：2

共 10 条

[1] An Open, Large-Scale, Collaborative Effort to Estimate the Reproducibility of Psychological Science [J].

Alexander, Anita ;

Barnett-Cowan, Michael ;

Bartmess, Elizabeth ;

Bosco, Frank A. ;

Brandt, Mark ;

Carp, Joshua ;

Chandler, Jesse J. ;

Clay, Russ ;

Cleary, Hayley ;

Cohn, Michael ;

Costantini, Giulio ;

DeCoster, Jamie ;

Dunn, Elizabeth ;

Eggleston, Casey ;

Estel, Vivien ;

Farach, Frank J. ;

Feather, Jenelle ;

Fiedler, Susann ;

Field, James G. ;

Foster, Joshua D. ;

Frank, Michael ;

Frazier, Rebecca S. ;

Fuchs, Heather M. ;

Galak, Jeff ;

Galliani, Elisa Maria ;

Garcia, Sara ;

Giammanco, Elise M. ;

Gilbert, Elizabeth A. ;

Giner-Sorolla, Roger ;

Goellner, Lars ;

Goh, Jin X. ;

Goss, R. Justin ;

Graham, Jesse ;

Grange, James A. ;

Gray, Jeremy R. ;

Gripshover, Sarah ;

Hartshorne, Joshua ;

Hayes, Timothy B. ;

Jahn, Georg ;

Johnson, Kate ;

Johnston, William ;

Joy-Gaba, Jennifer A. ;

Lai, Calvin K. ;

Lakens, Daniel ;

Lane, Kristin ;

LeBel, Etienne P. ;

Lee, Minha ;

Lemm, Kristi ;

Mackinnon, Sean ;

May, Michael .

PERSPECTIVES ON PSYCHOLOGICAL SCIENCE, 2012, 7 (06) :657-660

[2]

[Anonymous], 2015, SCIENCE, DOI [DOI 10.1126/science.aac4716, DOI 10.1126/SCIENCE.AAC4716]

[3]

Cumming G., 2004, Understanding Statistics, V3, P299, DOI [10.1207/s15328031us0304_5, DOI 10.1207/S15328031US0304_5]

[4] Confidence intervals and replication: Where will the next mean fall? [J].

Cumming, Geoff ;

Maillardet, Robert .

PSYCHOLOGICAL METHODS, 2006, 11 (03) :217-227

[5] Using prediction markets to estimate the reproducibility of scientific research [J].

Dreber, Anna ;

Pfeiffer, Thomas ;

Almenberg, Johan ;

Isaksson, Siri ;

Wilson, Brad ;

Chen, Yiling ;

Nosek, Brian A. ;

Johannesson, Magnus .

PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 2015, 112 (50) :15343-15347

[6]

Ebersole C. R., 2016, J EXP SOC PSYCHOL, V65

[7] Comment on "Estimating the reproducibility of psychological science" [J].

Gilbert, Daniel T. ;

King, Gary ;

Pettigrew, Stephen ;

Wilson, Timothy D. .

SCIENCE, 2016, 351 (6277)

[8] Investigating Variation in Replicability A "Many Labs'' Replication Project [J].

Klein, Richard A. ;

Ratliff, Kate A. ;

Vianello, Michelangelo ;

Adams, Reginald B., Jr. ;

Bahnik, Stepan ;

Bernstein, Michael J. ;

Bocian, Konrad ;

Brandt, Mark J. ;

Brooks, Beach ;

Brumbaugh, Claudia Chloe ;

Cemalcilar, Zeynep ;

Chandler, Jesse ;

Cheong, Winnee ;

Davis, William E. ;

Devos, Thierry ;

Eisner, Matthew ;

Frankowska, Natalia ;

Furrow, David ;

Galliani, Elisa Maria ;

Hasselman, Fred ;

Hicks, Joshua A. ;

Hovermale, James F. ;

Hunt, S. Jane ;

Huntsinger, Jeffrey R. ;

IJzerman, Hans ;

John, Melissa-Sue ;

Joy-Gaba, Jennifer A. ;

Kappes, Heather Barry ;

Krueger, Lacy E. ;

Kurtz, Jaime ;

Levitan, Carmel A. ;

Mallett, Robyn K. ;

Morris, Wendy L. ;

Nelson, Anthony J. ;

Nier, Jason A. ;

Packard, Grant ;

Pilati, Ronaldo ;

Rutchick, Abraham M. ;

Schmidt, Kathleen ;

Skorinko, Jeanine L. ;

Smith, Robert ;

Steiner, Troy G. ;

Storbeck, Justin ;

Van Swol, Lyn M. ;

Thompson, Donna ;

van 't Veer, A. E. ;

Vaughn, Leigh Ann ;

Vranka, Marek ;

Wichman, Aaron L. ;

Woodzicka, Julie A. .

SOCIAL PSYCHOLOGY, 2014, 45 (03) :142-152

[9] Registered Reports A Method to Increase the Credibility of Published Results [J].

Nosek, Brian A. ;

Lakens, Daniel .

SOCIAL PSYCHOLOGY, 2014, 45 (03) :137-141

[10] Shall We Really Do It Again? The Powerful Concept of Replication Is Neglected in the Social Sciences [J].

Schmidt, Stefan .

REVIEW OF GENERAL PSYCHOLOGY, 2009, 13 (02) :90-100

← 1 →