Extension of receiver operating characteristic analysis to data concerning multiple signal detection tasks

Rationale and Objectives. When receiver operating characteristic (ROC) studies involve multiple detection tasks in the same patient, the correlations among the within-patient interpretations and the shortage of patients raise several analytic challenges. The authors propose methods for both parametric and nonparametric analysis of this type of data. Methods. First, they computed one accuracy statistic for each task. Then, one jackknife ''pseudovalue'' was calculated for each task and each patient. Then, they formed a summary index from the accuracy statistics and calculated the standard error of this summary index on the basis of the observed correlations among the jackknife pseudovalues. The method was extended to comparisons of two diagnostic systems and illustrated by using data from a clinical study. Results and Conclusion. This approach can be used with rating or quantitative data and with any index of accuracy, whether calculated parametrically or nonparametrically.