Can we turn shirkers into workers?

被引:53
作者
Berinsky, Adam J. [1 ]
Margolis, Michele F. [2 ]
Sances, Michael W. [3 ]
机构
[1] MIT, Dept Polit Sci, 77 Massachusetts Ave,E53-457, Cambridge, MA 02139 USA
[2] Univ Penn, Dept Polit Sci, Philadelphia, PA 19104 USA
[3] Univ Memphis, Dept Polit Sci, Memphis, TN 38152 USA
关键词
Attention checks; Surveys; Validity; Screener; Noise; AMAZON MECHANICAL TURK; INTERNET; RESPONSES; PROJECT; ONLINE;
D O I
10.1016/j.jesp.2015.09.010
中图分类号
B84 [心理学];
学科分类号
04 ; 0402 ;
摘要
Survey researchers increasingly employ attention checks to identify inattentive respondents and reduce noise. Once inattentive respondents are identified, however, researchers must decide whether to drop such respondents, thus threatening external validity, or keep such respondents, thus threatening internal validity. In this article, we ask whether there is a third way: can inattentive respondents be induced to pay attention? Using three different strategies across three studies, we show that while such inducements increase attention check passage, they do not reduce noise in descriptive or experimental survey items. In addition, the inducements cause some respondents to drop out of the survey. These results have important implications for applied research. While scholars should continue to measure inattention via attention checks, increasing the attentiveness of "shirker" respondents is not as easy as previously thought. (C) 2015 Elsevier Inc. All rights reserved.
引用
收藏
页码:20 / 28
页数:9
相关论文
共 28 条
[1]   Separating the Shirkers from the Workers? Making Sure Respondents Pay Attention on Self-Administered Surveys [J].
Berinsky, Adam J. ;
Margolis, Michele F. ;
Sances, Michael W. .
AMERICAN JOURNAL OF POLITICAL SCIENCE, 2014, 58 (03) :739-753
[2]   Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk [J].
Berinsky, Adam J. ;
Huber, Gregory A. ;
Lenz, Gabriel S. .
POLITICAL ANALYSIS, 2012, 20 (03) :351-368
[3]   Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data? [J].
Buhrmester, Michael ;
Kwang, Tracy ;
Gosling, Samuel D. .
PERSPECTIVES ON PSYCHOLOGICAL SCIENCE, 2011, 6 (01) :3-5
[4]   Nonnaivete among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers [J].
Chandler, Jesse ;
Mueller, Pam ;
Paolacci, Gabriele .
BEHAVIOR RESEARCH METHODS, 2014, 46 (01) :112-130
[5]   National Surveys Via Rdd Telephone Interviewing Versus the Internet [J].
Chang, Linchiat ;
Krosnick, Jon A. .
PUBLIC OPINION QUARTERLY, 2009, 73 (04) :641-678
[6]   Web surveys - A review of issues and approaches [J].
Couper, MP .
PUBLIC OPINION QUARTERLY, 2000, 64 (04) :464-494
[7]   Evaluating Amazon's Mechanical Turk as a Tool for Experimental Behavioral Research [J].
Crump, Matthew J. C. ;
McDonnell, John V. ;
Gureckis, Todd M. .
PLOS ONE, 2013, 8 (03)
[8]   Methods for the detection of carelessly invalid responses in survey data [J].
Curran, Paul G. .
JOURNAL OF EXPERIMENTAL SOCIAL PSYCHOLOGY, 2016, 66 :4-19
[9]   Detecting and Deterring Insufficient Effort Responding to Surveys [J].
Huang, Jason L. ;
Curran, Paul G. ;
Keeney, Jessica ;
Poposki, Elizabeth M. ;
DeShon, Richard P. .
JOURNAL OF BUSINESS AND PSYCHOLOGY, 2012, 27 (01) :99-114
[10]   Reconceptualizing replication as a sequence of different studies: A replication typology [J].
Hueffmeier, Joachim ;
Mazei, Jens ;
Schultze, Thomas .
JOURNAL OF EXPERIMENTAL SOCIAL PSYCHOLOGY, 2016, 66 :81-92