Interobserver agreement for 10% categories of angiographic carotid stenosis

Background and Purpose Although the reliability of the assessment of severe 70% to 99% carotid stenosis by carotid angiography has been proven excellent, this may not necessarily be the case for a more detailed classification of carotid stenoses by 10% categories. Methods Angiograms of the carotid arteries were assessed pairwise by three independent, experienced observers. The measurements of the degree of stenosis of both the carotid bifurcation and the internal carotid artery were made according to the European Carotid Surgery Trial method. Kappa statistics were used to assess the agreement beyond chance for severe (70% to 99%) carotid stenosis (kappa(1)) and for 10% categories of carotid stenosis (kappa(2)). The penalty scores were adjusted by weights for the relative difference in risk (RDR) of stroke in the ipsilateral carotid distribution between the 10% categories (kappa(3)). An adjustment of the RDR method was made by assuming that only patients with a severe carotid stenosis would undergo surgery, and the penalty would be 0 if no disagreement would exist about the indication for surgery (kappa(4)). An even further adjustment (kappa(5)) was made by assuming that assessment of the rate of carotid stenosis by one or both observers would lead to different treatment recommendations in 50% of the cases, and accordingly the penalty for disagreement (RDR) was halved. Results One hundred twenty-one carotid bifurcations in 65 patients with a transient ischemic attack or nondisabling stroke were assessed. The intraclass correlation between the exact estimates of carotid stenosis was .90 (95% confidence interval, .85 to .92). The mean difference in stenosis between the two raters was 0.8% (95% confidence interval, -2.1% to 3.7%). kappa(1) to kappa(5) equaled 0.80, 0.40, 0.79, 0.91, and 0.92 respectively. Conclusions Interobserver agreement for distinct 10% categories of angiographic carotid stenosis is moderate, but when realistic risk-and decision-based weights are used, agreement between experienced observers can be almost perfect.