Model comparisons and model selections based on generalization criterion methodology

The purpose of this article is to formalize the generalization criterion method for model comparison. The method has the potential to provide powerful comparisons of complex and nonnested models that may also differ in terms of numbers of parameters. The generalization criterion differs From the better known cross-validation criterion in the following critical procedure. Although both employ a calibration stage to estimate parameters, cross-validation employs a replication sample from the same design for the validation stage, whereas generalization employs a new design for the critical stage. Two examples of the generalization criterion method are presented that demonstrate its usefulness for selecting a model based on sound scientific principles out of a set that also contains models lacking sound scientific principles that are either overly complex or oversimplified. The main advantage of the generalization criterion is its reliance on extrapolations to new conditions. After all, accurate a priori predictions to new conditions are the hallmark of a good scientific theory. (C) 2000 Academic Press.