Quantitative summaries of single-subject studies: What do group comparisons tell us about individual performances?

Kollins, Newland, and Critchfield (1999) responded to our comments about their review by arguing that their quantitative summary was not a meta-analysis and should not be criticized in these terms. We reply that regardless of what they call their review, it included confounding effects that make interpretations of the results problematic. Kollins et al. also argued that unexpected findings of the sort they reported can serve as a spur for further research. We reply that the understanding of findings that deviate from existing knowledge may well require empirical investigation. Such endeavors, however, should begin with an evaluation of the review procedures that suggested the existence of the differences. Finally, we emphasize that quantitative summaries of individual data are, in the end, a form of group comparison. The implications of using group methods to clarify individual data deserve frank recognition in discussions of the outcomes.