Gauging the impact of growing nonresponse on estimates from a national RDD telephone survey

Declining contact and cooperation rates in random digit dial (RDD) national telephone surveys raise serious concerns about the validity of estimates drawn from such research. While research in the 1990s indicated that nonresponse bias was relatively small, response rates have continued to fall since then. The current study replicates a 1997 methodological experiment that compared results from a "Standard" 5-day survey employing the Pew Research Center's usual methodology with results from a "Rigorous" survey conducted over a much longer field period and achieving a significantly higher response rate. As with the 1997 study, there is little to suggest that unit nonresponse within the range of response rates obtained seriously threatens the quality of survey estimates. In 77 out of 84 comparable items, the two surveys yielded results that were statistically indistinguishable. While the "Rigorous" study respondents tended to be somewhat less politically engaged, they did not report consistently different behaviors or attitudes on other kinds of questions. With respect to sample composition, the Standard survey was closely aligned with estimates from the U.S. Census and other large government surveys on most variables. We extend our analysis of nonresponse to include comparisons with the hardest-to-reach respondents and with respondents who terminated the interview prior to completion.