Recent advances in observer performance methodology: Jackknife free-response ROC (JAFROC)

The jackknife free-response receiver operating characteristic (JAFROC) method allows quantitative analysis of observer data such as that observed when radiologists interpret images, which could contain more than one lesion and a location can be reported for each perceived lesion. The method was recently validated with a perception-based simulation model that incorporated the detectability parameter of the standard binormal ROC model. and in addition allowed simultaneous samples from both noise and signal distributions. The total number of noise samples is an important new parameter that measures reader expertise. The new sampling model incorporates search, which is an integral part of lesion detection that has not been possible to model until now. The model was used to generate simulated FROC ratings, data. which was used to assess the statistical validity of JAFROC analysis. We found that JAFROC analysis is a statistically valid approach for analysing FROC data and that JAFROC analysis exhibited significantly greater statistical power than the existing ROC approach.