BIAS IN VALIDITY GENERALIZATION VARIANCE ESTIMATES - REPLY

In a recent article Thomas (1988) derived the expected value of the true validity variance estimate used in validity generalization studies. Based on computations of the expected values for certain scenarios, Thomas made a number of critical assertions regarding the variance estimate. This article shows that Thomas's arguments regarding deficiencies in the variance estimate used in validity generalization studies are misleading. Contrary to Thomas' extremely negative assessment of the situation, there is no really convincing reason to doubt or abandon the estimates of true validity variance obtained in applied research from the Callender-Osburn and other closely related methods. Rather, there is strong evidence to indicate that populations of true validities with meaningful differences in mean and variance can be reliably distinguished, provided that a sufficient amount of base data are available.