An empirical comparison of statistical construct validation approaches

The use of measurement instruments to examine causal relationships among constructs constituting theoretical frameworks is important to advancing engineering management research. This paper examines two broad implementation approaches to statistical refinement and validation of measurement instruments. The two approaches differ in their refinement procedures in their use of principal component factor analysis (Approach A) and conventional confirmatory factor analysis (Approach B). It is difficult to evaluate the net impact of these fundamental differences between the two approaches on the resulting statistical construct validity merely using theoretical arguments. To assess their power of construct refinement and validation, we undertook a comparison of the outcomes of the two approaches using two measurement instruments (the TQM instrument and the Supervisor instrument). In addition, we tested the potential benefits of blending the two approaches into a third "Hybrid Approach." Results indicate that Approach B and the Hybrid Approach provide refined scales with higher unidimensionality, reliability, convergent validity, and discriminant validity. However, Approach A and the Hybrid Approach can identify and split constructs with underlying patterns indicating existence of multiple dimensions and yield better operationalization of the nomological framework. In conclusion, the Hybrid Approach combines the strengths of Approach A and Approach B. It performs well not only in terms of the statistical validity of constructs, but also incorporates the feature to recognize patterns suggested by exploratory methods. We recommend its use for refining and validating measurement instruments in relatively unexplored research domains as well as in matured research domains. The results have strong applicability for statistical construct validation of instruments in engineering management and other fields using measurement instruments.