Observer agreement in the angiographic assessment of arteriovenous malformations of the brain

Purpose-We aimed to determine intraobserver and interobserver agreement in the characterization of brain arteriovenous malformation (AVM) angioarchitecture on intra-arterial digital subtraction angiograms. Methods-Five experienced interventional neuroradiologists independently reviewed 40 anonymized angiograms obtained at the time of first-ever AVM diagnosis. The allocation of the films to observers was balanced for AVM size and complexity. Every observer was compared with himself and all the others by distributing the films in 2 batches 3 months apart. The observers used standard forms to collect both quantitative and categorized qualitative angiographic data. To measure agreement we used the kappa statistic (K) for nominal data, weighted K for ordinal and discrete interval data, and Bland & Altman analysis for continuous data. Results-Intraobserver agreement was generally moderate to substantial, with 95% confidence intervals ranging from fair to almost perfect. However, for every characteristic, interobserver agreement was less than intraobserver agreement. Interobserver agreement was generally slight to moderate, with 95% confidence intervals ranging from less than chance to almost perfect. Conclusion-This study demonstrates the need for robust and generalizeable definitions of AVM angioarchitecture and methods of nidus size measurement-with proof of good intraobserver and interobserver agreement-for future efforts to understand the prognosis and best treatment of AVMs.