RELIABILITY AND CONSTRUCT-VALIDITY OF A STRUCTURED TECHNICAL SKILLS ASSESSMENT FORM

Current methods of evaluating technical competence of surgical residents are subjective and potentially unreliable. This study assesses the reliability and construct validity of a new format for the assessment of technical ability, the two part Structured Technical Skills Assessment Form (STSAF). Part I, which is completed while an operation is proceeding consists of approximately 120 essential components of the procedure, Part II, completed at the end of the operation, is a 10-point global rating form. Forty-one operations were evaluated using the STSAF, with multiple observers present at 26. Inter-rater reliability of both Parts I and II were high (.78 and .73, respectively). Statistically significant differences were noted between senior-resident and junior-resident performances, suggesting construct validity. The incorporation of structured guidelines to the assessment of technical skill leads to high inter-rater reliability and construct validity, which ultimately may result in improved and reproducible evaluations of surgical trainees.