Interobserver variations in assessment of antral disease from direct sinoscopic observations compared to video recordings

We tested the reliability of visual clinical sinoscopic observations by using video recordings. An edited video tape was made from the video-recorded bilateral antral sinoscopies of nine patients included in a prospective case series of inflammatory maxillary sinus disease in critically ill patients. An expert panel of four consultant ENT surgeons made individual appraisals of the video using a structured form. They had only been informed in general about the study, but had no further case knowledge in order to avoid all biases. Concordance and interobserver variabilities in the expert panel were compared in pairs. The assessments of the expert panel were compared with the observations of the performer, who was biased through individual case knowledge. Nine patients, either nasotracheally intubated or tracheotomized, had ventilator treatment for more than 7 days. One ENT surgeon performed all sinoscopies and made the direct observations. Satisfactory agreement was found on mucosal alterations. The concordance of recognizing bacterial infection was almost perfect. However; the assessment of redness from a video recording displayed only chance agreement. The differences between kappa statistic values within the expert panel group compared to the comparisons of performer versus expert panel were not significant. The case knowledge of the sinoscopist did not alter the assessments markedly and thus seemed only to have minor influence. Video recording as a method for second opinion was shown to be a useful technique, but will have to be further developed.