The power metric: a new statistically robust enrichment-type metric for virtual screening applications with early recovery capability

被引：27

作者：

Dias Lopes, Julio Cesar ^{[1
]}

dos Santos, Fabio Mendes ^{[1
]}

Martins-Jose, Andrelly ^{[1
]}

Augustyns, Koen ^{[2
]}

De Winter, Hans ^{[2
]}

机构：

[1] Univ Fed Minas Gerais, Dept Quim, Chemoinformat Grp, NEQUIM, Belo Horizonte, MG, Brazil

[2] Univ Antwerp, Dept Pharmaceut Sci, Med Chem Grp, Campus Drie Eiken,Bldg A,Univ Pl 1, B-2610 Antwerp, Belgium

来源：

JOURNAL OF CHEMINFORMATICS | 2017年 / 9卷

关键词：

Power metric (PM); Virtual screening; Metric; Model performance; Enrichment factor; Area under the curve (AUC); Receiver operating curve enrichment factor (ROCE); Correct classification rate (CCR); Matthews correlation coefficient (MCC); Cohen's kappa coefficient (CKC); Relative enrichment factor (REF); AGREEMENT; PERFORMANCE; YOUDEN;

D O I：

10.1186/s13321-016-0189-4

中图分类号：

O6 [化学];

学科分类号：

070301 [无机化学];

摘要：

A new metric for the evaluation of model performance in the field of virtual screening and quantitative structure-activity relationship applications is described. This metric has been termed the power metric and is defined as the fraction of the true positive rate divided by the sum of the true positive and false positive rates, for a given cutoff threshold. The performance of this metric is compared with alternative metrics such as the enrichment factor, the relative enrichment factor, the receiver operating curve enrichment factor, the correct classification rate, Matthews correlation coefficient and Cohen's kappa coefficient. The performance of this new metric is found to be quite robust with respect to variations in the applied cutoff threshold and ratio of the number of active compounds to the total number of compounds, and at the same time being sensitive to variations in model quality. It possesses the correct characteristics for its application in early-recognition virtual screening problems.

引用

页数：11

共 31 条

[1]

DIAGNOSTIC-TESTS-2 - PREDICTIVE VALUES .4. [J].

ALTMAN, DG ;

BLAND, JM .

BRITISH MEDICAL JOURNAL, 1994, 309 (6947) :102-102

[2]

[Anonymous], 2003, Statistical Methods for Rates and Proportions

[3]

Peirce, Youden, and receiver operating characteristic curves [J].

Baker, Stuart G. ;

Kramer, Barnett S. .

AMERICAN STATISTICIAN, 2007, 61 (04) :343-346

[4]

About the relationship between ROC curves and Cohen's kappa [J].

Ben-David, Arie .

ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE, 2008, 21 (06) :874-882

[5]

Comparison of classification accuracy using Cohen's Weighted Kappa [J].

Ben-David, Arie .

EXPERT SYSTEMS WITH APPLICATIONS, 2008, 34 (02) :825-832

[6]

Brodersen Kay H., 2010, Proceedings of the 2010 20th International Conference on Pattern Recognition (ICPR 2010), P3121, DOI 10.1109/ICPR.2010.764

[7]

Carletta J, 1996, COMPUT LINGUIST, V22, P249

[8]

A COEFFICIENT OF AGREEMENT FOR NOMINAL SCALES [J].

COHEN, J .

EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 1960, 20 (01) :37-46

[9]

Comparison of Several Molecular Docking Programs: Pose Prediction and Virtual Screening Accuracy [J].

Cross, Jason B. ;

Thompson, David C. ;

Rai, Brajesh K. ;

Baber, J. Christian ;

Fan, Kristi Yi ;

Hu, Yongbo ;

Humblet, Christine .

JOURNAL OF CHEMICAL INFORMATION AND MODELING, 2009, 49 (06) :1455-1474

[10]

An introduction to ROC analysis [J].

Fawcett, Tom .

PATTERN RECOGNITION LETTERS, 2006, 27 (08) :861-874

← 1 2 3 4 →