Computer-aided lung nodule detection in CT: Results of large-scale observer test

Rationale and Objectives. The objective is to study the incremental effects of using a computer-aided lung nodule detection (CAD) system on the performance of a large pool of observers. Materials and Methods. A set of eight thin-section computed tomographic data sets with limited longitudinal coverage, containing a total of 22 lung nodules, was analyzed by using the automated nodule detection system. When applied to all eight cases, the CAD system alone achieved a detection rate of 86.4%, with 2.64 false-positive results per case. This study included 202 observers at a national radiology meeting: 39 thoracic radiologists, 95 nonthoracic radiologists, and 68 nonradiologists. Each participant read from one to eight cases in random order, first without and then with CAD system output available. Observer performance in nodule detection was measured before and after CAD was made available. Differences in performance of groups of observers before and after CAD were tabulated by mean, median, and SD in detection rate and number of false-positive results and tested by using nonparametric methods. Results. In an analysis involving only the first randomly selected case read by all 202 participants, there were statistically significant increases in nodule detection rates and numbers of false-positive results for all types of observers. There was a significant difference in detection rates between radiologists and nonradiologists before CAD, but after CAD, there was no significant difference in detection rates between these observer types. In a second analysis involving 13 participants who read all eight cases. mean detection rates were 64.0% before CAD and 81.9% after CAD. Mean numbers of false-positive results were 0.144 per case before CAD and 0.173 after CAD. Conclusion. In a large observer study, use of a CAD system for nodule detection resulted in an incremental increase in detection rate, but also led to an increase in number of false-positive results. Also, CAD appears to be an equalizer of detection rates between observers of different levels of experience.