AVERAGING JUDGES RATINGS TO INCREASE THEIR CORRELATION WITH AN EXTERNAL CRITERION

It is well-known that the ratings of judges become more reliable as more judges' ratings are averaged. Furthermore, more reliable measures are expected to be more valid. However, empirical data do not exist which show that the averaged ratings of more judges are in fact more valid. In this study, 2 groups of 10 judges (clinicians and students) rated a patient's mood from speech samples taken at different times during psychotherapy. The judges' ratings were averaged and then correlated with an objective measure of anxiety based upon the patient's speech disruptions. The correlations did increase (up to a theoretical asymptote) as the number of judges increased. (22 ref) (PsycINFO Database Record (c) 2006 APA, all rights reserved). © 1979 American Psychological Association.