Estimation of denominator degrees of freedom of F-distributions for assessing Wald statistics for fixed-effect factors in unbalanced mixed models

Tests for fixed-effect factors in unbalanced mixed models have previously used t-tests on a contrast-by-contrast basis or Wald statistics without a universally accepted method of calculating the denominator degrees of freedom. This situation has arisen because the variances of different contrasts are differently weighted sums of the variance components with associated degrees of freedom that are not necessarily equal. A simultaneous F-test for differences between all levels of a fixed-effect factor can be derived by forming new contrasts, by rotation of the original contrasts, with variances that are close to being the same weighted sum of variance components. The associated degrees of freedom for these new contrasts are nearly equal. A small simulation study shows the appropriateness of a X-2 approximation to the distribution of the weighted sums of variance components. Three simple examples are used to demonstrate the effects of rotation. The last of these examples is also used to compare the proposed simultaneous F-test with the distribution of the Wald statistic obtained by numerical simulation. The method of rotations is then applied to data on the range size of mountain hares (Lepus timidus) to assess the evidence for a two-way interaction between season and habitat.