OBSERVER VARIATION IN THE INTERPRETATION OF LOWER-LIMB VENOGRAMS

After agreeing on diagnostic criteria and after a pilot study, two experienced radiologists twice independently reviewed 40 lower limb venograms performed by a standard technique in patients suspected or known to have venous thrombosis. The observers reviewed 20 examinations at a time, their analysis requiring separate identification of 11 major veins. At each site observers stated whether thrombus was 'absent,' 'doubtful,' 'presumed,' or 'definite,' or declared 'no opinion possible.' They then rediscussed criteria of diagnosis and, using the same experimental design, examined another 40 venograms. To correct for agreement expected by chance, data were analyzed by using the kappa statistic. In general, levels of agreement were higher than those reported for many other clinical and radiologic investigations, probably because of refinement of criteria after the pilot study. Nonetheless, observers disagreed about the probable presence or absence of thrombus at some site in the limb in about 10% of examinations. Observer variation should be considered when venography is used as a reference standard to evaluate other methods of diagnosing thrombi.