Jiving the Four-Step, Waltzing Around Factor Analysis, and Other Serious Fun

It has been proposed that a clear separation of measurement from structural reasons for model failure can be obtained via a procedure testing 4 nested models: (a) a factor model, (b) a confirmatory factor model, (c) the anticipated structural equation model, and (d) possibly, a more constrained model. Advocates of the 4-step procedure contend that these nested models provide a trustworthy way of determining whether one's model is failing as a result of structural (conceptual) inadequacy, or as a result of measurement misspecification. We argue that measurement and structural issues can not be unambiguously separated by the 4 steps, and that the seeming separation is incomplete at best and illusory at worst. The prime difficulty is that the 4-step procedure is incapable of determining whether the proposed model contains the proper number of factors. As long as the number of factors is in doubt, measurement and structural assessments remain dubious and entwined. The assessment of model fit raises additional difficulties because the researcher is implicitly favoring of the null hypothesis, and the logic of the root mean square error of approximation (RMSEA) as a test of "close fit" is inconsistent with the logic of the 4-step. These discussions question whether factor analysis can dependably determine the proper number of factors, and argue against the routine use of .05 as the probability target for structural equation model chi-square fit.