POWER AND TYPE-I ERROR RATES OF NEW PAIRWISE MULTIPLE COMPARISON PROCEDURES UNDER HETEROGENEOUS VARIANCES

The Type I error rates and statistical power of 9 selected multiple comparison procedures were compared in a Monte Carlo study. Data were generated for 3-, 4-, and 5-group ANOVA models, from simulated populations with both homogeneous and heterogeneous variances, A variety of patterns of population means were examined, including completely null, partial-null, and multiple-null patterns. None of the procedures was robust to violations of the assumption of variance homogeneity at nominal alpha levels lower than.10, even with equal sample sizes. The Dunn procedure and modified Bonferroni procedures showed better robustness properties than the Tukey procedure and recent modifications of it did. Power comparisons, conducted at a nominal alpha level of.10, showed the Peritz, Ryan, and Fisher-Hayter tests to be consistently most powerful across the variance conditions examined. However, power differences among these three procedures were minimal.