Testing repeated measures hypotheses when covariance matrices are heterogeneous: Revisiting the robustness of the Welch-James test

This article examines the recommendations given by Keselman, Carriere and Lix (1993) regarding choice of sample size for obtaining robust tests of the repeated measures main and interaction hypotheses in a one Between-Subjects by one Within-Subjects design with a Welch-James type multivariate test when covariance matrices were heterogeneous and data were non-normal in unbalanced designs. We examined the generalizability of their recommendations by varying the (a) size of the design, (b) degree of covariance heterogeneity, and (c) degree of nonsphericity. Our results indicate that the Keselman et al. recommendations for the test of the repeated measures main effect hold in some situations and can be relaxed in others. Consequently, with a relatively modest sample size, the Welch-James test provides a robust test of the main effect. On the other hand, the required sample size for the interaction can be quite large; accordingly, the Keselman et al. recommendations must be increased as the number of groups increases and when the data are skewed. We recommend the Welch-James test for testing the main effect hypothesis. The Welch-James test should also be used to test the interaction hypothesis when the sample sizes are sufficiently large to permit a robust test. In other conditions, the researcher should use an alternative such as Huynh's (1978) Improved General Approximation test.