Inter- and intra-observer variation in classification systems for impending fractures of bone metastases

The study was designed to assess the reproducibility and reliability of Mirels' scoring system and the conventional scoring system for impending pathological fractures. The results of both classification systems influence the choice of therapeutic procedures offered to these patients. Eight independent observers (four orthopaedic surgeons and four radiologists with varying clinical experience) scored blinded plain radiographs from 47 patients with bone metastases. Each observer scored the radiographs as per the Mirels and the conventional systems. After 12 weeks, the observers scored the radiographs again. Inter- and intra-observer agreement was assessed based on the weighted kappa coefficient values for both systems. For intra-observer reproducibility, kappa values for the conventional system had a mean of 0.499 (SD 0.074) showing a moderate agreement, while Mirels' scoring system had a mean of 0.396 (SD 0.101) showing a fair agreement. For inter-observer reliability, kappa values for the conventional scoring system were 0.322 for the first test and 0.47 for the second test, giving fair and moderate agreement respectively. For Mirels' scoring system, the kappa coefficient for inter-observer reliability was 0.183 for the first test and 0.218 for the second, giving poor and fair agreement respectively. The conventional scoring system showed better inter and intra-observer agreement compared with Mirels' scoring system. Both systems fail to take into account factors such as co-morbidities and prognosis. We believe the conventional system is a good screening tool, but a new scoring system is required for impending pathological fractures.