A systematic analysis of performance measures for classification tasks

被引：3750

作者：

Sokolova, Marina ^{[1
]}

Lapalme, Guy ^{[2
]}

机构：

[1] Childrens Hosp Eastern Ontario, Elect Hlth Informat Lab, Ottawa, ON K1H 8L1, Canada

[2] Univ Montreal, Dept Informat & Rech Operat, Montreal, PQ H3C 3J7, Canada

来源：

INFORMATION PROCESSING & MANAGEMENT | 2009年 / 45卷 / 04期

基金：

加拿大自然科学与工程研究理事会;

关键词：

Performance evaluation; Machine Learning; Text classification;

D O I：

10.1016/j.ipm.2009.03.002

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

This paper presents a systematic analysis of twenty four performance measures used in the complete spectrum of Machine Learning classification tasks, i.e., binary, multi-class, multi-labelled, and hierarchical. For each classification task, the study relates a set of changes in a confusion matrix to specific characteristics of data. Then the analysis concentrates on the type of changes to a confusion matrix that do not change a measure, therefore, preserve a classifier's evaluation (measure invariance). The result is the Measure invariance taxonomy with respect to all relevant label distribution changes in a classification problem. This formal analysis is supported by examples of applications where invariance properties of measures lead to a more reliable evaluation of classifiers. Text classification Supplements the discussion with several case studies. (C) 2009 Elsevier Ltd. All rights reserved.

引用

页码：427 / 437

页数：11

共 46 条

[1]

[Anonymous], J MACHINE LEARNING R

[2]

[Anonymous], 2002, Proceedings of the ICML-2002 Workshop on Data Mining Lessons Learned

[3]

[Anonymous], 2006, P 2006 C EMP METH NA

[4]

[Anonymous], 2005, P 28 ANN INT ACM SIG, DOI DOI 10.1145/1076034.1076082

[5]

[Anonymous], 1996, Elements of Machine Learning

[6]

Asuncion A., 2007, UCI Machine Learning Repository

[7]

BENGIO S, 2005, P ICML 05 WORKSH ROC, P43

[8]

Blitzer John., 2007, Annual Meeting-Association For Computational Linguistics, V45, P440

[9]

BOBICEV V, 2008, P 23 NAT C ART INT A, P1444

[10]

Cohen J., 1988, Statistical Power Analysis for the Behavioral Sciences, V2nd, DOI [10.4324/9780203771587, DOI 10.4324/9780203771587]

← 1 2 3 4 5 →