Reproducibility -: an important factor determining the quality of computer aided detection (CAD) systems

Purpose: To test the reproducibility of markings on mammography films set by a commercial computer aided detection (CAD) system. Patients and methods: One hundred unilateral mammography examinations (each in CC and MLO) of 100 patients with mammographically detected suspicious foci, which were histopathologically proven to be malignant, were scanned three times with the CAD system. retrospectively. Every fifth patient of the institutional tumor case sampler was enrolled in the study. Only cases with one visible lesion were included in the study. Reproducibility and sensitivity (in both the strict and the broader sense) were determined. Strict sensitivity means the correct set of markers in both images, whereas broader sensitivity means the correct set in at least one of the images. Sixteen of 100 malignancies were indicated by focal suspicious microcalcification clusters, 53 tumors by masses and 31 cases by both signs of breast cancer. The CAD evaluation was divided into only two different markers: one for microcalcifications and one fur masses. Thus, 47 (16 +/- 31) tumor-induced microcalcifications and 84 (53 + 31) malignancy-related masses were checked using the CAD system. Results: Eighteen of 100 unilateral mammography examinations revealed identical patterns in all three scans (18% reproducibility). Eleven of 47 suspicious focal microcalcification clusters and 43/84 masses were correctly marked on both mammographic views in all three CAD scans (strict and broader sensitivity, 23.4 and 51.1%, respectively). Six of 47 microcalcification clusters and 8/84 masses were totally missed in all images by the system (false negative rate. 12.8 and 9.6%, respectively). Conclusion: Reproducibility is essential for CAD systems. Currently, reproducibility of the used CAD system appears to be insufficient for clinical routine. Improvement of the system characteristics would make such systems valuable as a 'second reader' in clinical examination. (C) 2000 Elsevier Science Ireland Ltd. All rights reserved.