Bootstrap calibration to improve the reliability of tests to compare sample means and variances

The comparison of several sample means to see whether they differ significantly is a common analysis, which is not straightforward when the samples may be from non-normal distributions with different variances. A recent study found that a randomization test that attempts to approximate the distribution of F-statistics from one- and two-factor analysis of variance in the presence of unequal population variances was the best of 12 alternative tests considered. However, it sometimes suffered from excess size with data from extremely non-normal distributions. In the present article a method for improving the robustness of the test by bootstrap calibration is described for one-factor analysis of variance, and shown to be effective by a simulation study. The method is also applied with Levene's test for unequal variance by randomization. In this case the test is very robust without calibration, and calibration does not improve it. Copyright (C) 2001 John Wiley & Sons, Ltd.