Comparison of bench test evaluations of surgical skill with live operating performance assessments

BACKGROUND: Attempts at assessing surgical proficiency have generally used laboratory simulation to evaluate skill. The aim of this study was to compare technical ability as measured on a bench simulation with actual operative performance. STUDY DESIGN: Twenty-two general surgeons and trainees were recruited: consultants (n = 4), specialist registrars (n = 14), and senior house officers (n = 4). They were assessed while performing a saphenofemoral dissection on an anesthetized patient in the operating theater, and performing the same procedure on an inanimate model within the laboratory. The Objective Structured Assessment of Technical Skill method, consisting of a 7-parameter global rating (maximum score 35) and 17-point step-by-step checklist (maximum score 17) was used to measure performance in both environments. Face, content, and construct validity of the synthetic model were established as part of this study. RESULTS: There was a significant relationship between technical skill as measured on the bench test model and performance within the operating theater with respect to both global rating (Spearman correlation coefficient 0.824, p < 0.001; alpha coefficient 0.89) and checklist ratings (r = 0.514, p < 0.02; alpha coefficient 0.68) rating assessments. Global rating scores correlated with experience for both operative (r = 0.822, p < 0.001) and bench (r = 0.515, p < 0.05) settings. There was no difference in level of measured performance between operating theater and bench model Global rating mean 23.25 +/- 6.66 versus 23.75 +/- 5.62, respectively; paired t-test p = 0.559). CONCLUSIONS: Assessment of technical skill using inanimate procedural simulation translates to actual surgical performance within the operating theater. This further validates use of bench test evaluations to measure surgical technical ability (C) 2004 by the American College of Surgeons.