What's wrong with hit ratio?

被引:7
作者
Ben-David, Arie [1 ]
机构
[1] Holon Inst Technol, Holon, Israel
关键词
D O I
10.1109/MIS.2006.123
中图分类号
TP18 [人工智能理论];
学科分类号
081104 [模式识别与智能系统]; 0812 [计算机科学与技术]; 0835 [软件工程]; 1405 [智能科学与技术];
摘要
The views of author on issues related to faults in hit ratio and its magnitude are presented. He has evaluated the magnitude through an empirical experiment on three multivalued classification data sets, using two machine learning models. The concept of hit ratio is problematic as it doesn't compensate for classifications caused by chance. ROC curves offer an alternative to using hit ratio are useful visualization tools for analyzing trade-offs between true positives and fast positives. A classifier that does better than randon yields a concave ROC curve above that straight line. By computing the area between these two lines, the classifiers performance can be estimated as relative to a random sample. ROC curves convey more information than possible via a singular scalar metrix. The results confirm that using hit ratio for assessing a classifier accuracy in machine learning includes many successes that could have resulted from chance.
引用
收藏
页码:68 / 70
页数:3
相关论文
共 7 条
[1]
[2]
Cook RJ., 1998, ENCY BIOSTATISTICS, P2166
[3]
HIGH AGREEMENT BUT LOW KAPPA .1. THE PROBLEMS OF 2 PARADOXES [J].
FEINSTEIN, AR ;
CICCHETTI, DV .
JOURNAL OF CLINICAL EPIDEMIOLOGY, 1990, 43 (06) :543-549
[4]
MISINTERPRETATION AND MISUSE OF THE KAPPA-STATISTIC [J].
MACLURE, M ;
WILLETT, WC .
AMERICAN JOURNAL OF EPIDEMIOLOGY, 1987, 126 (02) :161-169
[5]
Margineantu D.D., 1997, Pruning adaptive boosting, V97, P211
[6]
Provost F., 1997, Proceedings of the Third International Conference on Knowledge Discovery and Data Mining, P43
[7]
A REAPPRAISAL OF THE KAPPA-COEFFICIENT [J].
THOMPSON, WD ;
WALTER, SD .
JOURNAL OF CLINICAL EPIDEMIOLOGY, 1988, 41 (10) :949-958