The importance of complexity in model selection

Model selection should be based not solely on goodness-of-fit, but must also consider model complexity. While the goal of mathematical modeling in cognitive psychology is to select one model from a set of competing models that best captures the underlying mental process. choosing the model that best fits a particular set of data will not achieve this goal. This is because a highly complex model can provide a good lit without necessarily bearing any interpretable relationship with he underlying process. it is shown that model selection based solely on the fit to observed data will result in the choice of all unnecessarily complex model that overfits the data, and thus generalizes poorly. The effect of over-fitting must be properly of first by model selection methods. An application example of selection methods using artificial data is also presented. (C) 2000 Academic Press.