A reliable and valid instrument to assess competency in basic surgical skills in second-year medical students

Background: Despite calls for competency based education, a dearth of validated instruments for measuring basic skills currently exists. We developed an instrument to assess competency in basic surgical skills in second-year medical students and tested it for psychometric reliability and validity. Methods: From a review of the literature, an instrument comprised of numerically scaled items was constructed. After initial tests, several items were divided to produce a final instrument more specific and more appropriate for providing feedback to students. The final instrument was empirically tested for reliability and validity. Results: The final 10-item instrument is presented here along with all of the empirical evidence including internal consistency reliability and interrater reliability, and content, criterion-related, and construct validity. Overall alpha reliability was 0.84 and interrater reliability was r = 0.83, P < 0.01 for the total scores. Factor analysis provided evidence of construct validity. Conclusions: The instrument has psychometric properties adequate for use as one criterion for summative evaluation and is educationally practical enough to provide focused and detailed feedback for student improvement.