About the relationship between ROC curves and Cohen's kappa

被引：137

作者：

Ben-David, Arie ^{[1
]}

机构：

[1] Holon Inst Technol, Dept Technol Management, IL-58102 Holon, Israel

来源：

ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE | 2008年 / 21卷 / 06期

关键词：

classification accuracy; ROC curves; area under ROC curve (AUC); Cohen's kappa; machine learning;

D O I：

10.1016/j.engappai.2007.09.009

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 [计算机科学与技术];

摘要：

Receiver operating characteristic (ROC) curves are very powerful tools for measuring classifiers' accuracy in binary-class problems. However, their usefulness in real-world multi-class problems has not been demonstrated yet. In these frequently occurring multi-class cases, simple accuracy meters that do compensate for random successes, such as the kappa statistic, are needed. ROC curves are two-dimensional graphs. Kappa is a scalar. Each comes from an entirely different discipline. This research investigates whether they do have anything in common. A mathematical formulation that links ROC spaces with the kappa statistic is derived here for the first time. The understanding of how these two accuracy meters relate to each other can assist in a better understanding of their respective pros and cons. (c) 2007 Elsevier Ltd. All rights reserved.

引用

页码：874 / 882

页数：9

共 22 条

[1]

Alpaydin Ethem, 2004, Introduction to machine learning

[2]

[Anonymous], 2005, Data Mining Pratical Machine Learning Tools and Techniques

[3]

[Anonymous], 1998, ICML

[4]

A lot of randomness is hiding in accuracy [J].