Using operative outcome to assess technical skill

BACKGROUND: This study examined whether an operative product and time to completion could serve as measures of technical skill. METHODS: Nine final-year (PGY5) and 11 penultimate-year (PGY4) general surgery residents participated in a 6-station bench model examination. Time to completion was recorded. Twelve faculty surgeons (2 per station) evaluated the quality of the final product using a 5-point scale. RESULTS: The mean interrater reliability was 0.59 for product quality. Interstation reliability was 0.59 for analysis of the final product and 0.72 for time to completion. There was 63% and 78% agreement between attendings' ratings and product quality and time scores respectively. PGY5s' mean product quality score was 4.14 +/- 0.26, compared with 3.82 +/- 0.33 for PGY4s (P < 0.05). PGY5s' mean time was 110 +/- 19 minutes compared with PGY4s' 132 +/- 15 (P < 0.05). CONCLUSIONS: Analysis of the operative end product and time to completion offer efficient alternatives to on-line examiner scoring for bench model examinations of technical competence. Am J Surg. 2000;180:234-237. (C) 2000 by Excerpta Medica, Inc.