A comparison of Type I error rates and power levels for seven solutions to the multivariate Behrens-Fisher problem

Several alternatives to Hotelling's T-2 have been recommended for the case where variance-covariance matrices are unknown and unequal. Type I error rates and power were estimated for Hotelling's T-2 and for seven such solutions, including Bennett's (1951), James' (1954), Yao's (1965), Johansen's (1980), Nel and Van der Merwe's (1986), Hwang and Paulson's R-B (1986), and Kim's (1992). Additionally, the power for each solution was calculated after adjusting for Type I error rates. The number of variables, the level of intercorrelation present among the variates, the degree of heteroscedasticity, and the sample sizes associated with each of the two mean vectors were varied in each of 36 factor combinations. For each factor combination, 10,000 repetitions were run. James' procedure almost always had the highest power, but its Type I error rate was almost always greater than the nominal. Kim's and Nel and Van der Merwe's procedures had the highest power among procedures whose Type I error rates were not inflated. Type I error rates for Hotelling's T-2 were almost always inflated, even when sample sizes were equal.