Pilot study of a 360-degree assessment instrument for physical medicine and rehabilitation residency programs

Objective: To perform a pilot test on a new format for multidisciplinary assessment of resident physicians' professionalism and clinical performance in acute inpatient rehabilitation settings. Design: In this pilot study, a 26-item ratings instrument was developed for use by therapists, nurses, social workers, case managers, and psychologists to rate inpatient residents. Results: A total of 421 ratings forms were returned over four academic years. Alpha reliability coefficient for instrumentation sample was 0.99. chi(2) and analysis of variance procedures examined item mean differences. Significant differences (P less than or equal to 0.05) were found based on resident sex (17 items) and rotation setting (20 items). No significant differences were found based on rater profession; mean ratings by profession ranged from 6.67 (physical therapists) to 7.46 (case managers). Conclusions: Psychometric properties of this new ratings format are encouraging. The tool was a useful way to provide formative feedback to residents regarding professionalism and performance. Residency program directors can use this approach to fulfill Accreditation Council for Graduate Medical Education mandates to use a variety of assessment methods regarding resident education. However, potential sex bias and other issues affecting performance ratings should be considered in interpreting results and warrant further study.