Citation Patterns Following a Strongly Contradictory Replication Result: Four Case Studies From Psychology

被引:24
作者
Hardwicke, Tom E. [1 ,2 ]
Szucs, Denes [3 ]
Thibault, Robert T. [4 ,5 ]
Cruwell, Sophia [2 ,6 ]
van den Akker, Olmo R. [7 ]
Nuijten, Michele B. [7 ]
Ioannidis, John P. A. [2 ,8 ,9 ,10 ,11 ,12 ]
机构
[1] Univ Amsterdam, Dept Psychol, Amsterdam, Netherlands
[2] Charite Univ Med Berlin, Meta Res Innovat Ctr Berlin METRIC B, QUEST Ctr Transforming Biomed Res, Berlin, Germany
[3] Univ Cambridge, Dept Psychol, Cambridge, England
[4] Univ Bristol, Sch Psychol Sci, Bristol, England
[5] Univ Bristol, MRC Integrat Epidemiol Unit, Bristol, England
[6] Univ Cambridge, Dept Hist & Philosophy Sci, Cambridge, England
[7] Tilburg Univ, Tilburg Sch Social & Behav Sci, Dept Methodol & Stat, Tilburg, Netherlands
[8] Stanford Univ, Meta Res Innovat Ctr Stanford METRICS, Stanford, CA 94305 USA
[9] Stanford Univ, Dept Med, Stanford, CA 94305 USA
[10] Stanford Univ, Dept Epidemiol & Populat Hlth, Stanford, CA 94305 USA
[11] Stanford Univ, Dept Biomed Data Sci, Stanford, CA 94305 USA
[12] Stanford Univ, Dept Stat, Stanford, CA 94305 USA
基金
欧洲研究理事会; 欧盟地平线“2020”;
关键词
replication; citations; meta-research; self-correction; citation bias; open data; open materials; preregistered; EGO-DEPLETION; EXPOSURE;
D O I
10.1177/25152459211040837
中图分类号
B84 [心理学];
学科分类号
04 ; 0402 ;
摘要
Replication studies that contradict prior findings may facilitate scientific self-correction by triggering a reappraisal of the original studies; however, the research community's response to replication results has not been studied systematically. One approach for gauging responses to replication results is to examine how they affect citations to original studies. In this study, we explored postreplication citation patterns in the context of four prominent multilaboratory replication attempts published in the field of psychology that strongly contradicted and outweighed prior findings. Generally, we observed a small postreplication decline in the number of favorable citations and a small increase in unfavorable citations. This indicates only modest corrective effects and implies considerable perpetuation of belief in the original findings. Replication results that strongly contradict an original finding do not necessarily nullify its credibility; however, one might at least expect the replication results to be acknowledged and explicitly debated in subsequent literature. By contrast, we found substantial citation bias: The majority of articles citing the original studies neglected to cite relevant replication results. Of those articles that did cite the replication but continued to cite the original study favorably, approximately half offered an explicit defense of the original study. Our findings suggest that even replication results that strongly contradict original findings do not necessarily prompt a corrective response from the research community.
引用
收藏
页数:14
相关论文
共 54 条
[1]   Estimating the reproducibility of psychological science [J].
Aarts, Alexander A. ;
Anderson, Joanna E. ;
Anderson, Christopher J. ;
Attridge, Peter R. ;
Attwood, Angela ;
Axt, Jordan ;
Babel, Molly ;
Bahnik, Stepan ;
Baranski, Erica ;
Barnett-Cowan, Michael ;
Bartmess, Elizabeth ;
Beer, Jennifer ;
Bell, Raoul ;
Bentley, Heather ;
Beyan, Leah ;
Binion, Grace ;
Borsboom, Denny ;
Bosch, Annick ;
Bosco, Frank A. ;
Bowman, Sara D. ;
Brandt, Mark J. ;
Braswell, Erin ;
Brohmer, Hilmar ;
Brown, Benjamin T. ;
Brown, Kristina ;
Bruening, Jovita ;
Calhoun-Sauls, Ann ;
Callahan, Shannon P. ;
Chagnon, Elizabeth ;
Chandler, Jesse ;
Chartier, Christopher R. ;
Cheung, Felix ;
Christopherson, Cody D. ;
Cillessen, Linda ;
Clay, Russ ;
Cleary, Hayley ;
Cloud, Mark D. ;
Cohn, Michael ;
Cohoon, Johanna ;
Columbus, Simon ;
Cordes, Andreas ;
Costantini, Giulio ;
Alvarez, Leslie D. Cramblet ;
Cremata, Ed ;
Crusius, Jan ;
DeCoster, Jamie ;
DeGaetano, Michelle A. ;
Della Penna, Nicolas ;
den Bezemer, Bobby ;
Deserno, Marie K. .
SCIENCE, 2015, 349 (6251)
[2]  
[Anonymous], 1992, Changing order: Replication and induction in scientific practice
[3]  
Aust F., 2020, papaja: Create APA manuscripts with R Markdown [Manual]
[4]   Citation Distortions in the Literature on the Serotonin-Transporter-Linked Polymorphic Region and Amygdala Activation [J].
Bastiaansen, Jojanneke A. ;
de Vries, Ymkje Anna ;
Munafo, Marcus R. .
BIOLOGICAL PSYCHIATRY, 2015, 78 (08) :E35-E36
[5]   Ego depletion: Is the active self a limited resource? [J].
Baumeister, RF ;
Bratslavsky, E ;
Muraven, M ;
Tice, DM .
JOURNAL OF PERSONALITY AND SOCIAL PSYCHOLOGY, 1998, 74 (05) :1252-1265
[6]   Misguided Effort With Elusive Implications [J].
Baumeister, Roy F. ;
Vohs, Kathleen D. .
PERSPECTIVES ON PSYCHOLOGICAL SCIENCE, 2016, 11 (04) :574-575
[7]   The psychology of experimental psychologists: Overcoming cognitive constraints to improve research: The 47th Sir Frederic Bartlett Lecture [J].
Bishop, Dorothy V. M. .
QUARTERLY JOURNAL OF EXPERIMENTAL PSYCHOLOGY, 2020, 73 (01) :1-19
[8]   The Replication Recipe: What makes for a convincing replication? [J].
Brandt, Mark J. ;
IJzerman, Hans ;
Dijksterhuis, Ap ;
Farach, Frank J. ;
Geller, Jason ;
Giner-Sorolla, Roger ;
Grange, James A. ;
Perugini, Marco ;
Spies, Jeffrey R. ;
van 't Veer, Anna .
JOURNAL OF EXPERIMENTAL SOCIAL PSYCHOLOGY, 2014, 50 :217-224
[9]   Phenomena of retraction - Reasons for retraction and citations to the publications [J].
Budd, JM ;
Sievert, ME ;
Schultz, TR .
JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, 1998, 280 (03) :296-297
[10]   A Series of Meta-Analytic Tests of the Depletion Effect: Self-Control Does Not Seem to Rely on a Limited Resource [J].
Carter, Evan C. ;
Kofler, Lilly M. ;
Forster, Daniel E. ;
McCullough, Michael E. .
JOURNAL OF EXPERIMENTAL PSYCHOLOGY-GENERAL, 2015, 144 (04) :796-815