Analysis and design of multitrait-multirater performance appraisal studies

Becker and Cote (1994) found that the correlated uniqueness model outperformed the confirmatory factor analysis and direct product models for multitrait-multimethod data. The present study analyzed 20 multitrait-multirater performance appraisal matrices. The correlated uniqueness model was appropriate significantly more often than in Pecker and Cote's study and the other two models performed poorly. The proportions of trait and method variance in ratings were related to several rating system characteristics such as opportunity for raters to observe ratees and basing rating dimensions on a job analysis. Performance of all three models was better with larger proportions of trait variance and smaller proportions of method variance.