INFORMATION-BASED EVALUATION CRITERION FOR CLASSIFIERS PERFORMANCE

被引：55

作者：

KONONENKO, I ^{[1
]}

BRATKO, I ^{[1
]}

机构：

[1] JOZEF STEFAN INST,LJUBLJANA,YUGOSLAVIA

来源：

MACHINE LEARNING | 1991年 / 6卷 / 01期

关键词：

CLASSIFIER; EVALUATION CRITERIA; MACHINE LEARNING; INFORMATION THEORY;

D O I：

10.1023/A:1022642017308

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

In the past few years many systems for learning decision rules from examples were developed. As different systems allow different types of answers when classifying new instances, it is difficult to appropriately evaluate the systems' classification power in comparison with other classification systems or in comparison with human experts. Classification accuracy is usually used as a measure of classification performance. This measure is, however, known to have several defects. A fair evaluation criterion should exclude the influence of the class probabilities which may enable a completely uninformed classifier to trivially achieve high classification accuracy. In this paper a method for evaluating the information score of a classifier's answers is proposed. It excludes the influence of prior probabilities, deals with various types of imperfect or probabilistic answers and can be used also for comparing the performance in different domains.

引用

页码：67 / 80

页数：14

共 19 条

[11] Quinlan J. R., 1986, Machine Learning, V1, P81, DOI 10.1023/A:1022643204877
[12] Quinlan J. R., 1979, EXPERT SYSTEMS MICRO
[13] RIPLEY KL, 1975, COMPUT CARDIOL, P27
[14] Shannon Claude E., 1949, MATH THEORY COMMUNIC
[15] SPACKMAN KA, 1989, 6TH P INT WORKSH MAC, P160
[16] WEISS SM, 1987, 6TH P ANN NAT C ART, P521
[17] WILLIAMS BT, 1982, COMPUTER AIDS CLIN D, V1
[18] WILLIAMS BT, 1982, COMPUTER AIDS CLIN D, V2
[19] COMPUTER ASSESSMENT OF OBSERVER PERFORMANCE BY RECEIVER OPERATING CHARACTERISTIC CURVE AND INFORMATION-THEORY
WINTER, J
[J]. COMPUTERS AND BIOMEDICAL RESEARCH, 1982, 15 (06): : 555 - 562

← 1 2 →