Assessing interrater agreement on the job relevance of a test:: A comparison of the CVI, T, rWG(J), and r*WG(J) indexes

This article examines the problem of estimating interrater agreement about the job relevance of test items by examining 4 different indexes: C. H. Lawshe's (1975) Content Validity Index (CVI); H. E. A. Tinsley and D. J. Weiss's (1975) T; L. R. James, R. G. Demaree, and G. Wolf's (1984) r(WG(J)); and M. K. Lindell, C. J. Brandt, and D. J. Whitney's (1999) r*(WG(J)). These 4 indexes are shown to be systematically related to each other, but r*(WG(J)) is computationally simpler and is more generally applicable than the other indexes.