Neural network ensembles: evaluation of aggregation algorithms

被引:124
作者
Granitto, PM [1 ]
Verdes, PF [1 ]
Ceccatto, HA [1 ]
机构
[1] Univ Nacl Rosario, CONICET, Inst Fis Rosario, RA-2000 Rosario, Santa Fe, Argentina
关键词
machine learning; ensemble methods; neural networks; regression;
D O I
10.1016/j.artint.2004.09.006
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Ensembles of artificial neural networks show improved generalization capabilities that outperform those of single networks. However, for aggregation to be effective, the individual networks must be as accurate and diverse as possible. An important problem is, then, how to tune the aggregate members in order to have an optimal compromise between these two conflicting conditions. We present here an extensive evaluation of several algorithms for ensemble construction, including new proposals and comparing them with standard methods in the literature. We also discuss a potential problem with sequential aggregation algorithms: the non-frequent but damaging selection through their heuristics of particularly bad ensemble members. We introduce modified algorithms that cope with this problem by allowing individual weighting of aggregate members. Our algorithms and their weighted modifications are favorably tested against other methods in the literature, producing a sensible improvement in performance on most of the standard statistical databases used as benchmarks. (c) 2004 Elsevier B.V. All rights reserved.
引用
收藏
页码:139 / 162
页数:24
相关论文
共 24 条
[11]  
Freund Y., 1995, COMPUTATIONAL LEARNI, V904, P23, DOI DOI 10.1007/3-540-59119-2_166
[12]  
Friedman J, Towards an expansive epistemology: Norms, action and the social sphere
[13]   NEURAL NETWORKS AND THE BIAS VARIANCE DILEMMA [J].
GEMAN, S ;
BIENENSTOCK, E ;
DOURSAT, R .
NEURAL COMPUTATION, 1992, 4 (01) :1-58
[14]   Women's primary health care and the search for quality [J].
Grant, JM .
WOMENS HEALTH ISSUES, 2001, 11 (04) :305-309
[15]   MULTIPLE-VALUED STATIONARY STATE AND ITS INSTABILITY OF THE TRANSMITTED LIGHT BY A RING CAVITY SYSTEM [J].
IKEDA, K .
OPTICS COMMUNICATIONS, 1979, 30 (02) :257-261
[16]  
Karakoulas G, 2000, ADV NEUR IN, P247
[17]  
Müller A, 1999, CLIN DIAGN LAB IMMUN, V6, P243
[18]   Optimal ensemble averaging of neural networks [J].
Naftaly, U ;
Intrator, N ;
Horn, D .
NETWORK-COMPUTATION IN NEURAL SYSTEMS, 1997, 8 (03) :283-296
[19]  
NAVONE H, 2001, REV IBEROAMERICANA I, V3, P70
[20]   Sparse regression ensembles in infinite and finite hypothesis spaces [J].
Rätsch, G ;
Demiriz, A ;
Bennett, KP .
MACHINE LEARNING, 2002, 48 (1-3) :189-218