How to measure the diagnostic accuracy of noninvasive liver fibrosis indices: The area under the ROC curve revisited

BACKGROUND: The area under the ROC curve (AUC) is widely used to measure the diagnostic accuracy of noninvasive fibrosis indices. However, use of the AUC assumes a binary gold standard, whereas fibrosis staging is based on an ordinal scale and also depends on the distribution of fibrosis stages in the study sample. We explored other fibrosis staging accuracy measures designed for ordinal gold standards, the C-statistic and the Obuchowski measure. METHODS: We performed a simulation study to assess the bias in estimating the accuracy measures when the distribution of fibrosis stages in the study sample do not fit the reference distribution in the population to which the indices are applied. We also estimated the type I error of the tests comparing these measures in 2 samples with different distributions of fibrosis stages. We illustrated the practical use of these measures by reanalyzing real data. RESULTS: Compared with the AUC or the C-statistic, the Obuchowski measure showed limited bias when the distribution of fibrosis stages in the study sample differed from the reference distribution. The type I error was strongly inflated with the AUC or the C-statistic but was preserved in the Obuchowski measure. When we compared noninvasive indices on real data, AUC analysis led to discordant results depending on how the fibrosis stages were grouped together. One single conclusion was drawn from the analysis based on the Obuchowski measure. CONCLUSIONS: We recommend using the Obuchowski measure for assessing the diagnostic accuracy of noninvasive indices of fibrosis. (C) 2008 American Association for Clinical Chemistry.