Methods of model accuracy estimation can help selecting the best models from decoy sets: Assessment of model accuracy estimations in CASP11

被引:43
作者
Kryshtafovych, Andriy [1 ]
Barbato, Alessandro [2 ,3 ]
Monastyrskyy, Bohdan [1 ]
Fidelis, Krzysztof [1 ]
Schwede, Torsten [2 ,3 ]
Tramontano, Anna [4 ]
机构
[1] Univ Calif Davis, Genome Ctr, 451 Hlth Sci Dr, Davis, CA 95616 USA
[2] Univ Basel, Biozentrum, Basel, Switzerland
[3] Swiss Inst Bioinformat, Basel, Switzerland
[4] Sapienza Univ Rome, Dept Phys, Rome, Italy
关键词
CASP; EMA; QA; estimation of model accuracy; model quality assessment; protein structure modeling; protein structure prediction; PROTEIN-STRUCTURE PREDICTION; QUALITY ASSESSMENT; SERVER; SCORE;
D O I
10.1002/prot.24919
中图分类号
Q5 [生物化学]; Q7 [分子生物学];
学科分类号
071010 ; 081704 ;
摘要
The article presents assessment of the model accuracy estimation methods participating in CASP11. The results of the assessment are expected to be useful to both- developers of the methods and users who way too often are presented with structural models without annotations of accuracy. The main emphasis is placed on the ability of techniques to identify the best models from among several available. Bivariate descriptive statistics and ROC analysis are used to additionally assess the overall correctness of the predicted model accuracy scores, the correlation between the predicted and observed accuracy of models, the effectiveness in distinguishing between good and bad models, the ability to discriminate between reliable and unreliable regions in models, and the accuracy of the coordinate error self-estimates. A rigid-body measure (GDT_TS) and three local-structure-based scores (LDDT, CADaa, and SphereGrinder) are used as reference measures for evaluating methods' performance. Consensus methods, taking advantage of the availability of several models for the same target protein, perform well on the majority of tasks. Methods that predict accuracy on the basis of a single model perform comparably to consensus methods in picking the best models and in the estimation of how accurate is the local structure. More groups than in previous experiments submitted reasonable error estimates of their own models, most likely in response to a recommendation from CASP and the increasing demand from users. (C) 2015 Wiley Periodicals, Inc.
引用
收藏
页码:349 / 369
页数:21
相关论文
共 17 条
[1]   QMEAN server for protein model quality estimation [J].
Benkert, Pascal ;
Kuenzli, Michael ;
Schwede, Torsten .
NUCLEIC ACIDS RESEARCH, 2009, 37 :W510-W514
[2]   MolProbity: all-atom structure validation for macromolecular crystallography [J].
Chen, Vincent B. ;
Arendall, W. Bryan, III ;
Headd, Jeffrey J. ;
Keedy, Daniel A. ;
Immormino, Robert M. ;
Kapral, Gary J. ;
Murray, Laura W. ;
Richardson, Jane S. ;
Richardson, David C. .
ACTA CRYSTALLOGRAPHICA SECTION D-STRUCTURAL BIOLOGY, 2010, 66 :12-21
[3]   Assessment of predictions in the model quality assessment category [J].
Cozzetto, Domenico ;
Kryshtafovych, Andriy ;
Ceriani, Michele ;
Tramontano, Anna .
PROTEINS-STRUCTURE FUNCTION AND BIOINFORMATICS, 2007, 69 :175-183
[4]   Evaluation of CASP8 model quality predictions [J].
Cozzetto, Domenico ;
Kryshtafovych, Andriy ;
Tramontano, Anna .
PROTEINS-STRUCTURE FUNCTION AND BIOINFORMATICS, 2009, 77 :157-166
[5]   VERIFY3D: Assessment of protein models with three-dimensional profiles [J].
Eisenberg, D ;
Luthy, R ;
Bowie, JU .
MACROMOLECULAR CRYSTALLOGRAPHY, PT B, 1997, 277 :396-404
[6]   Assessment of the assessment: Evaluation of the model quality estimates in CASP10 [J].
Kryshtafovych, Andriy ;
Barbato, Alessandro ;
Fidelis, Krzysztof ;
Monastyrskyy, Bohdan ;
Schwede, Torsten ;
Tramontano, Anna .
PROTEINS-STRUCTURE FUNCTION AND BIOINFORMATICS, 2014, 82 :112-126
[7]   CASP prediction center infrastructure and evaluation measures in CASP10 and CASP ROLL [J].
Kryshtafovych, Andriy ;
Monastyrskyy, Bohdan ;
Fidelis, Krzysztof .
PROTEINS-STRUCTURE FUNCTION AND BIOINFORMATICS, 2014, 82 :7-13
[8]   Evaluation of model quality predictions in CASP9 [J].
Kryshtafovych, Andriy ;
Fidelis, Krzysztof ;
Tramontano, Anna .
PROTEINS-STRUCTURE FUNCTION AND BIOINFORMATICS, 2011, 79 :91-106
[9]   Protein structure prediction and model quality assessment [J].
Kryshtafovych, Andriy ;
Fidelis, Krzysztof .
DRUG DISCOVERY TODAY, 2009, 14 (7-8) :386-393
[10]   lDDT: a local superposition-free score for comparing protein structures and models using distance difference tests [J].
Mariani, Valerio ;
Biasini, Marco ;
Barbato, Alessandro ;
Schwede, Torsten .
BIOINFORMATICS, 2013, 29 (21) :2722-2728