Multiple-reader studies, digital mammography, computer-aided diagnosis - and the Holy Grail of imaging physics (I)

There are multiple sources of variability in clinical studies of imaging systems. The variation of the reader "mindset" establishes the need for ROC analysis to control for that fundamental variable. The demonstration of the range of reader skills in mammography shows the need for a multivariate approach to ROC analysis. The multiple-reader, multiple-case (MRMC) ROC experimental paradigm addresses this need and several practical solutions to the problem of analysis of MRMC data have been developed. We review the application of these methods to an important clinical comparison of digital and conventional mammography. In a companion paper we review a more general problem-in which unaided readers of mammographic images are compared with readers aided by a computer processing and data fusion algorithm. The analysis there will suggest that the computer assist may allow incremental improvements in imaging physics and technology to make a clinically observable difference-by reducing the masking effect of reader variability.