A methodology for objective assessment of errors: an example using an endoscopic sinus surgery simulator

There is a public and professional demand for more performance accountability for surgeons, especially in the area of procedural proficiency. Little has been done in the area of objectively assessing technical skills, with the current standard being archived case load and surgeon observation with subjective commentary. Military and airline pilots have used flight simulators for over 50 years, but until recently surgeons were not tested in a similar manner. The Endoscopic Sinus Surgery Simulator has been developed for this purpose, and the initial validation has been published for skills performance. This article is devoted to describing the methodology for (1) the definition of abilities, skills, tasks and procedures, specifically concerning errors; (2) the taxonomy of errors in endoscopic sinus surgery with appropriate objective metrics; and (3) measures for outcomes analysis based on errors and other skills-performance measures. Although these metrics are developed specifically for the ES3, the method is generalizable for other otolaryngologic and general surgical procedures.