Performance of two-way ANOVA procedures when cell frequencies and variances are unequal

A fixed effect two-way ANOVA model with unequal cell frequencies and unequal error variances is considered. Under the Neyman-Pearson theory, exact tests for testing the interaction effect and main effect do not exist for this problem. When variances are unequal, classical F-tests which are calculated under the equal error variance assumption will provide only approximate solutions. For testing the interaction effect, we compare the performance of the generalized F-test and the classical F-test. Generalized F-test (generalized p-value) is a recently developed exact test which is based on an extended definition of the p-values. Using simulation, size., power and robustness comparisons are made. According to the simulation study, when heteroscedasticity is present under the normality, the size of the generalized F-test does not exceed the intended level allthough the size of the classical F-test exceeds the intended level. When the size is adjusted at the same level, for all the studied cases, the power of the generalized test is nearly equal or better than the power of the classical F-test. Both procedures are quite robust in terms of the size when heteroscedasticity is present under non-normality. Gamma distribution was used for non-normal comparisons. When the size is adjusted, for all the considered cases, the power of the generalized F-test is as good or better than the power of the classical F-test.