Scientific versus statistical inference

We argue that the goals of scientists in data analysis and scientific communication do not match the logic of hypothesis testing as it is typically taught in introductory statistics courses. The "Intro Stats" method has a number of well-known logical fallacies and the results of the procedures are often misinterpreted. However, we suspect that these problems do not directly compromise researchers inferences concerning their data because researchers implicitly use a different method for identifying the implications of data. In particular, we argue that researchers are typically interested in comparing the relative adequacy of different accounts of theoretical explanations rather than rejecting an artificially constructed null hypothesis; a simple analysis of recent journal articles supports this view. An alternative approach to data analysis and scientific communication is to present the strength of the evidence provided by an experiment in the form of a maximum likelihood ratio. This approach is more consistent with the logic of comparing alternative accounts that scientists commonly use. Likelihood ratios are simple to calculate and provide a simple and intuitive summary of the results relevant to evaluating the alternative explanations. Further, reporting likelihood ratios avoids the logical fallacies and interpretational problems of the Intro Stats method, allows one to identify compelling evidence for null effects, provides a simple method for evaluating failures to replicate, and can be readily used to aggregate results across experiments.