The differential receiver operating characteristic (DROC) method

The objective of this research was to develop a method for measuring small differences in diagnostic task performance between two imaging modalities. In the proposed Differential Receiver Operating Characteristic (DROC) method, the observer is shown a pair of images of the same patient, one from each modality A and B. The patient can be normal or abnormal but this information is not known to the observer. The observer selects the image that is preferred for the specified diagnostic task and assigns a rating. Analysis of this experiment yields the area under the DROC curve (A,). If A, is greater than 0.5, then modality B is superior to modality A and conversely, if it is less than 0.5, then A is superior to B. With a plausible assumption it can be shown that the result of the paired image presentation DROC experiment will track the difference in A(z)'s obtained from two single image presentation ROC experiments. A proof-of-concept experiment was conducted to test this idea. Both DROC and ROC studies were conducted using 6 readers. The DROC method was found to track the conventional ROC method and to yield far greater sensitivity, on the average about a factor of 3.46 greater. This work suggests a new paradigm for differential observer performance experiments. The DROC method has the potential for detecting extremely subtle differences in image quality, much smaller than is detectable by the present ROC method. This should allow rapid optimization of imaging systems, without the need for expensive and often inconclusive ROC studies.