Interrater reliability of assessing levator ani muscle defects with magnetic resonance images

The objective of this study is to determine interrater reliability of assessing pubovisceral levator ani muscle defects with magnetic resonance images. Normal pubovisceral muscle was assigned a grade of 0; PVM defects were graded as mild=1 (less than half missing), moderate=2 (more than half missing), and severe=3 (total or near total loss). Among six pairs of examiners, percent agreement and weighted kappa coefficients were calculated to determine agreement between pairs of examiners and among all examiners (i.e., "overall"). For unilateral scoring, exact agreement was found in 83.7%, and differences of one, two, and three grades were found in 14.7, 1.5, and 0.1%, respectively. For bilateral scoring, exact agreement and differences of one, two and three grades were found in 75.4, 15.9, 6.9, and 1.6%, respectively. Thus, exact agreement or a one-point difference was reached in 91.3% of cases. When defect status was categorized as none/normal, minor, and major, the overall weighted kappa coefficient was 0.86 (95% CI 0.83, 0.89). There was variation among examiner pairs with unilateral (p=0.002) and bilateral (p=0.02) scoring, but not when defect status was categorized as none/normal, minor, and major (p=0.59). There was agreement to within one point in 91% of cases when six examiner pairs scored levator ani defects on a seven-point scale. Examiner pairs discriminated injury similarly when defect status was categorized as normal/none, minor, or major.