The psychometric properties of 360 degree feedback: An empirical study and a cautionary tale

Amongst the purported advantages of multi-rater feedback systems is their greater accuracy and objectivity. It is argued here that these benefits may be more imagined than real, and that there is no reason to believe such systems will avoid many of the rating errors and distortions found in traditional top-down appraisal. The first study reported in this paper investigates the psychometric soundness and concurrent validity of a pilot 360 degree feedback scheme operated by a multi-national oil company. The second study describes how analysis of the data provided by the first study was used to re-design the rating form, and demonstrates the resulting improvement achieved in the psychometric properties of the 360 degree scheme. It also examined some of the variables that influence rater's assessment of the target managers. The conclusion is drawn that unless such feedback systems - irrespective of whether they are used for development or for appraisal - are constructed and evaluated along the lines associated with psychometric tests, they may produce assessments that are seriously misleading.