Is Competency Assessment at the Specialist Level Achievable? A Study for the National Training Programme in Laparoscopic Colorectal Surgery in England

被引:105
作者
Miskovic, Danilo [1 ]
Ni, Melody [1 ]
Wyles, Susannah M. [1 ]
Kennedy, Robin H. [2 ]
Francis, Nader K. [3 ]
Parvaiz, Amjad [4 ]
Cunningham, Chris [5 ]
Rockall, Timothy A. [6 ]
Gudgeon, Andrew M. [7 ]
Coleman, Mark G. [8 ]
Hanna, George B. [1 ]
机构
[1] Univ London Imperial Coll Sci Technol & Med, St Marys Hosp, Dept Surg & Canc, London W2 1NY, England
[2] St Marks Hosp, Harrow, Middx, England
[3] Yeovil Dist Hosp Fdn Trust Higher Kensington, Dept Surg, Yeovil, England
[4] Portsmouth Hosp NHS, Queen Alexandra Hosp, Dept Colorectal Surg, Portsmouth, Hants, England
[5] Oxford Radcliffe Hosp, Dept Colorectal Surg, Oxford, England
[6] Univ Surrey, MATTU, Postgrad Med Sch, Guildford GU2 5XH, Surrey, England
[7] Frimley Pk Hosp NHS Trust, Dept Surg, Frimley, Egypt
[8] Derriford Hosp, Natl Coordinat Off, Plymouth PL6 8DH, Devon, England
关键词
assessment; colorectal surgery; competency; education; laparoscopic surgery; training; ERRORS;
D O I
10.1097/SLA.0b013e318275b72a
中图分类号
R61 [外科手术学];
学科分类号
100210 [外科学];
摘要
Objectives: To develop, validate, and implement a competency assessment tool (CAT) for technical surgical performance in the context of a summative assessment process for the National Training Programme in Laparoscopic Colorectal Surgery (NTP). Background: The NTP is an educational initiative by the National Cancer Action Team in England to safely increase the uptake of laparoscopic colorectal surgery. It is the first competency-based national educational initiative for specialist surgeons (consultants), and performance assessment is an integral part of the program. Methods: Content validity was sought using expert opinion by semistructured interviews and the Delphi method. For validity and reliability studies, NTP apprentices and experts were asked to submit video-recorded cases. Construct validity was established between delegates who passed the assessment and those who failed. Concurrent validity was tested by comparing scores with error counts as identified by observational clinical human reliability analysis. A fully crossed design, using generalizability theory methods and D-studies, was used for reliability. Findings: Interviews and the Delphi method revealed a list of characteristics for assessment. A hybrid structure combining task-specific and generic items was used to include important characteristics into the assessment format. Fifty-four cases were submitted. Overall reliability reached G(ACI) = 0.803 when using 2 cases and 2 assessors. Experts scored significantly better than apprentices (3.19 vs 2.60; P = 0.004), and apprentices who passed had better scores than those who failed (2.95 vs 2.28; P < 0.001). There was an inverse correlation between CAT scores and observational clinical human reliability analysis error counts (rho = -0.520, P < 0.001). The combination of both methods reached overall sensitivity of 100%, specificity of 83.3%, a positive predictive value of 93.8%, and a negative predictive value of 100%. Conclusions: The CAT can reliably assess technical performance in laparoscopic colorectal surgery. The use of CATs to judge specialist technical performance before embarking on independent practice of new procedures is achievable on a national scale and can be adapted by other specialties.
引用
收藏
页码:476 / 482
页数:7
相关论文
共 25 条
[1]
Root cause analysis of internal hernia and Roux limb compression after laparoscopic Roux-en-Y gastric bypass using observational clinical human reliability assessment [J].
Ahmed, Ahmed R. ;
Miskovic, Danilo ;
Vijayaseelan, Thormela ;
O'Malley, William ;
Hanna, George B. .
SURGERY FOR OBESITY AND RELATED DISEASES, 2012, 8 (02) :158-163
[2]
Observational tools for assessment of procedural skills: a systematic review [J].
Ahmed, Kamran ;
Miskovic, Danilo ;
Darzi, Ara ;
Athanasiou, Thanos ;
Hanna, George B. .
AMERICAN JOURNAL OF SURGERY, 2011, 202 (04) :469-U161
[3]
Bloch R., 2010, G STRING 3 COMPUTER
[4]
QUALITATIVE RESEARCH .4. QUALITATIVE INTERVIEWS IN MEDICAL-RESEARCH [J].
BRITTEN, N .
BRITISH MEDICAL JOURNAL, 1995, 311 (6999) :251-253
[5]
Impact of established skills in open surgery on the proficiency gain process for laparoscopic surgery [J].
Brown, Daniel C. ;
Miskovic, Danilo ;
Tang, Benjie ;
Hanna, George B. .
SURGICAL ENDOSCOPY AND OTHER INTERVENTIONAL TECHNIQUES, 2010, 24 (06) :1420-1426
[6]
The National Training Programme for Laparoscopic Colorectal Surgery in England: a new training paradigm [J].
Coleman, M. G. ;
Hanna, G. B. ;
Kennedy, R. .
COLORECTAL DISEASE, 2011, 13 (06) :614-616
[7]
What do master surgeons think of surgical competence and revalidation? [J].
Cuschieri, A ;
Francis, N ;
Crosby, J ;
Hanna, GB .
AMERICAN JOURNAL OF SURGERY, 2001, 182 (02) :110-116
[8]
Dalkey NC., 1968, DELPHI METHOD METHOD
[9]
Medical education - Assessment in medical education [J].
Epstein, Ronald M. .
NEW ENGLAND JOURNAL OF MEDICINE, 2007, 356 (04) :387-396
[10]
CONSENSUS METHODS - CHARACTERISTICS AND GUIDELINES FOR USE [J].
FINK, A ;
KOSECOFF, J ;
CHASSIN, M ;
BROOK, RH .
AMERICAN JOURNAL OF PUBLIC HEALTH, 1984, 74 (09) :979-983