Statistical properties of the rWG(J) index of agreement

L. R. James, R. G. Demaree, and G. Wolf (1984) introduced r(WG(J)) to estimate interrater agreement for a group. This index is calculated by comparing an observed group variance with an expected random variance. As researchers have gained experience using this index, several questions have arisen. What are the consequences of replacing values beyond the unit interval by 0? What is the dependence of r(WG(J)) on the group size? The authors' simulations show that a positive bias is caused by the truncation, but for large population values of r(WG(J)) it is negligible. Also, in this case, the group size has no effect on the expected value of r(WG(J)). For inference on r(WR(J)), researchers can exploit the availability of computers to simulate data from the hypothesized distribution and then compare the simulation results for r(WG(J)) with the actual values. In addition, it is shown how the bootstrap method can be used for comparing the indices of 2 groups.