The reliability of EUS for the diagnosis of chronic pancreatitis: interobserver agreement among experienced endosonographers

Background: Endoscopic ultrasound (EUS) is a minimally invasive, low risk method of diagnosis for chronic pancreatitis (CP). The degree to which endosonographers agree on the features and diagnosis of CP is unknown. For EUS to be considered an accurate test for CP, there must be good interobserver agreement. Methods: Forty-five pancreatic EUS examinations were videotaped by 3 experienced endosonographers. Examinations from 33 patients with suspected CP based on typical symptoms, as well as 12 control patients without suspected CP, were included. Eleven experienced endosonographers ("experts") who were blinded to clinical information independently evaluated all videotaped examinations for the presence of CP and the following 9 validated features of CP: echogenic foci, strands, lobularity, cysts, stones, duct dilatation, duct irregularity, hyperechoic duct margins, and visible side branches. The experts also ranked (most to least) which features they believed to be the most indicative of CP. Interobserver agreement was expressed as the kappa (kappa) statistic. Results: There was moderately good overall agreement for the final diagnosis of CP (kappa = 0.45), Agreement was good for individual features of duct dilatation (kappa = 0.6) and lobularity (kappa = 0.51) but poor for the other 7 features (kappa < 0.4), The expert panel had consensus or near consensus agreement (greater than 90%) on 206 of 450 (46%) individual EUS features including 22 of 45 diagnoses of CP. Agreement on the final diagnosis of CP was moderately good for those trained in third tier fellowships (<kappa> = 0.42 +/- 0.03) and those with more than 1100 lifetime pancreatic EUS examinations (kappa = 0.46 +/- 0.05). The presence of stones was regarded as the most predictive feature of CP by all endosonographers, followed by visible side branches, cysts, lobularity, irregular main pancreatic duct, hyperechoic foci, hyperechoic strands, main pancreatic duct dilatation, and main duct hyperechoic margins. The most common diagnostic criterion for the diagnosis of CP was the total number of features (median 4 or greater, range 3 or greater to 5 or greater). Conclusions: EUS is a reliable method for the diagnosis of chronic pancreatitis with good interobserver agreement among experienced endosonographers. Agreement on the EUS diagnosis of chronic pancreatitis is comparable to other commonly used endoscopic procedures such as bleeding ulcer stigmata and computed tomography of the brain for stroke localization and better than the physical diagnosis of heart sounds.