Sample compositions and variabilities in published studies versus those in test manuals: Validity of score reliability inductions

As measurement specialists, we have done a disservice to both ourselves and our profession by habitually referring to "the reliability of the test," or saying that "the test is reliable." This has created a mind-set implying that reliability, once proven, is immutable. More important, practitioners and scholars need not know measurement theories if they may simply rely on the reliability purportedly intrinsic within all uses of established measures. The present study investigated empirically exactly how dissimilar in both composition and variability samples inducting reliability coefficients from prior studies were from the cited prior samples from which coefficients were generalized.