Classifier technology and the illusion of progress

被引:464
作者
Hand, David J.
机构
[1] Univ London Imperial Coll Sci Technol & Med, Dept Math, London SW7 2AZ, England
[2] Univ London Imperial Coll Sci Technol & Med, Inst Math Sci, London SW7 2AZ, England
关键词
supervised classification; error rate; misclassification rate; simplicity; principle of parsimony; population drift; selectivity bias; flat maximum effect; problem uncertainty; empirical comparisons;
D O I
10.1214/088342306000000060
中图分类号
O21 [概率论与数理统计]; C8 [统计学];
学科分类号
020208 [统计学]; 070103 [概率论与数理统计]; 0714 [统计学];
摘要
A great many tools have been developed for supervised classification, ranging from early methods such as linear discriminant analysis through to modern developments such as neural networks and support vector machines. A large number of comparative studies have been conducted in attempts to establish the relative superiority of these methods. This paper argues that these comparisons often fail to take into account important aspects of real problems, so that the apparent superiority of more sophisticated methods may be something of an illusion. In particular, simple methods typically yield performance almost as good as more sophisticated methods, to the extent that the difference in performance may be swamped by other sources of uncertainty that generally are not considered in the classical supervised classification paradigm.
引用
收藏
页码:1 / 14
页数:14
相关论文
共 46 条
[1]
Comparing classifiers when the misallocation costs are uncertain [J].
Adams, NM ;
Hand, DJ .
PATTERN RECOGNITION, 1999, 32 (07) :1139-1147
[2]
[Anonymous], 2003, Statistical pattern recognition
[3]
BENTON TC, 2002, THESIS IMPERIAL COLL
[4]
Statistical modeling: The two cultures [J].
Breiman, L .
STATISTICAL SCIENCE, 2001, 16 (03) :199-215
[5]
Cox DR, 2001, STAT SCI, V16, P216
[6]
A note on comparing classifiers [J].
Duin, RPW .
PATTERN RECOGNITION LETTERS, 1996, 17 (05) :529-536
[7]
Adaptive fraud detection [J].
Fawcett, T ;
Provost, F .
DATA MINING AND KNOWLEDGE DISCOVERY, 1997, 1 (03) :291-316
[8]
The use of multiple measurements in taxonomic problems [J].
Fisher, RA .
ANNALS OF EUGENICS, 1936, 7 :179-188
[10]
GALLAGHER JC, 1988, BONE MINER, V4, P189