Using smoothed receiver operating characteristic curves to summarize and compare diagnostic systems

Receiver operating characteristic (ROC) curves are used for summarizing the performance of imperfect diagnostic systems, especially in biomedical research. These curves are also appropriate for summarizing the performance of a discriminant analysis but are under-utilized by statisticians. This article is illustrates the use of these curves for comparing competing diagnostic systems, develops new estimation methods based on kernel density estimation, and studies the statistical performance of the new method. A transform of the ROC curve is further suggested based on the idea of "local population separation." This graphic is quite generally useful for displaying the differences between two populations. The methods are applied to a dataset comprising the results of seven diagnostics for predicting cancer activity on 353 patients. The distributions of these diagnostics are not well modeled parametrically, and so either completely nonparametric or kernel density estimation seems appropriate. Construction of the ROC curves shows not only if, but also how the diagnostics differ. The use of bootstrap simulation to check and adjust for the bias of smoothing is also demonstrated using these data.