An Evaluation of Amazon's Mechanical Turk, Its Rapid Rise, and Its Effective Use

被引:558
作者
Buhrmester, Michael D. [1 ]
Talaifar, Sanaz [2 ]
Gosling, Samuel D. [2 ,3 ]
机构
[1] Univ Oxford, Inst Cognit & Evolutionary Anthropol, 64 Banbury Rd, Oxford OX2 6PN, England
[2] Univ Texas Austin, Dept Psychol, Austin, TX 78712 USA
[3] Univ Melbourne, Melbourne Sch Psychol Sci, Melbourne, Vic, Australia
关键词
online methods; Internet research; Mechanical Turk; MTurk; QUALITY; PARTICIPANTS; WORKERS;
D O I
10.1177/1745691617706516
中图分类号
B84 [心理学];
学科分类号
04 ; 0402 ;
摘要
Over the past 2 decades, many social scientists have expanded their data-collection capabilities by using various online research tools. In the 2011 article "Amazon's Mechanical Turk: A new source of inexpensive, yet high-quality, data?" in Perspectives on Psychological Science, Buhrmester, Kwang, and Gosling introduced researchers to what was then considered to be a promising but nascent research platform. Since then, thousands of social scientists from seemingly every field have conducted research using the platform. Here, we reflect on the impact of Mechanical Turk on the social sciences and our article's role in its rise, provide the newest data-driven recommendations to help researchers effectively use the platform, and highlight other online research platforms worth consideration.
引用
收藏
页码:149 / 154
页数:6
相关论文
共 43 条
[1]  
[Anonymous], 2010, CHI EA
[2]   Comparisons of Online Recruitment Strategies for Convenience Samples: Craigslist, Google AdWords, Facebook, and Amazon Mechanical Turk [J].
Antoun, Christopher ;
Zhang, Chan ;
Conrad, Frederick G. ;
Schober, Michael F. .
FIELD METHODS, 2016, 28 (03) :231-246
[3]   The Importance of Assessing Clinical Phenomena in Mechanical Turk Research [J].
Arditte, Kimberly A. ;
Cek, Demet ;
Shaw, Ashley M. ;
Timpano, Kiara R. .
PSYCHOLOGICAL ASSESSMENT, 2016, 28 (06) :684-691
[4]   Can we turn shirkers into workers? [J].
Berinsky, Adam J. ;
Margolis, Michele F. ;
Sances, Michael W. .
JOURNAL OF EXPERIMENTAL SOCIAL PSYCHOLOGY, 2016, 66 :20-28
[5]   Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk [J].
Berinsky, Adam J. ;
Huber, Gregory A. ;
Lenz, Gabriel S. .
POLITICAL ANALYSIS, 2012, 20 (03) :351-368
[6]   Examining the relationship of perfectionism, depression, and optimism: Testing for mediation and moderation [J].
Black, Jessica ;
Reynolds, William M. .
PERSONALITY AND INDIVIDUAL DIFFERENCES, 2013, 54 (03) :426-431
[7]  
Buchanan T., 2010, Advanced Methods for Conducting Online Behavioral Research, P255, DOI DOI 10.1037/12076-016
[8]   Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data? [J].
Buhrmester, Michael ;
Kwang, Tracy ;
Gosling, Samuel D. .
PERSPECTIVES ON PSYCHOLOGICAL SCIENCE, 2011, 6 (01) :3-5
[9]  
Callegaro M, 2014, WILEY SER SURV METH, P1, DOI 10.1002/9781118763520
[10]   Separate but equal? A comparison of participants and data gathered via Amazon's MTurk, social media, and face-to-face behavioral testing [J].
Casler, Krista ;
Bickel, Lydia ;
Hackett, Elizabeth .
COMPUTERS IN HUMAN BEHAVIOR, 2013, 29 (06) :2156-2160