Simultaneous training of negatively correlated neural networks in an ensemble

被引:193
作者
Liu, Y [1 ]
Yao, X
机构
[1] Electrotech Lab, Div Comp Sci, Evolvable Syst Lab, Tsukuba, Ibaraki 3058568, Japan
[2] Univ Birmingham, Sch Comp Sci, Birmingham B15 2TT, W Midlands, England
来源
IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS PART B-CYBERNETICS | 1999年 / 29卷 / 06期
关键词
generalization; negative correlation learning; neural network ensembles;
D O I
10.1109/3477.809027
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
This paper presents a new cooperative ensemble learning system (CELS) for designing neural network ensembles. The idea behind CELS is to encourage different individual networks in an ensemble to learn different parts or aspects of a training data so that the ensemble can learn the whole training data better. In CELS, the individual networks are trained simultaneously rather than independently or sequentially. This provides an opportunity for the individual networks to interact with each other and to specialize. CELS can create negatively correlated neural networks using a correlation penalty term in the error function to encourage such specialization This paper analyzes CELS in terms of bias-variance-covariance tradeoff. CELS has also been tested on the Mackey-Glass time series prediction problem and the Australian credit card assessment problem. The experimental results show that CELS can produce neural network ensembles with good generalization ability.
引用
收藏
页码:716 / 725
页数:10
相关论文
共 20 条
[1]  
[Anonymous], 1993, Artificial Neural Networks for Speech and Vision
[2]   LIMITS FOR THE PRECISION AND VALUE OF INFORMATION FROM DEPENDENT SOURCES [J].
CLEMEN, RT ;
WINKLER, RL .
OPERATIONS RESEARCH, 1985, 33 (02) :427-442
[3]  
Cowder R.S., 1990, P 1990 CONN MOD SUMM, P117
[4]   BOOSTING AND OTHER ENSEMBLE METHODS [J].
DRUCKER, H ;
CORTES, C ;
JACKEL, LD ;
LECUN, Y ;
VAPNIK, V .
NEURAL COMPUTATION, 1994, 6 (06) :1289-1301
[5]   PREDICTING CHAOTIC TIME-SERIES [J].
FARMER, JD ;
SIDOROWICH, JJ .
PHYSICAL REVIEW LETTERS, 1987, 59 (08) :845-848
[6]   NEURAL NETWORKS AND THE BIAS VARIANCE DILEMMA [J].
GEMAN, S ;
BIENENSTOCK, E ;
DOURSAT, R .
NEURAL COMPUTATION, 1992, 4 (01) :1-58
[7]  
HAYKIN S, 1994, NEURAL NETWORKS COMP, P151
[8]   Bias/variance analyses of mixtures-of-experts architectures [J].
Jacobs, RA .
NEURAL COMPUTATION, 1997, 9 (02) :369-383
[9]   OSCILLATION AND CHAOS IN PHYSIOLOGICAL CONTROL-SYSTEMS [J].
MACKEY, MC ;
GLASS, L .
SCIENCE, 1977, 197 (4300) :287-288
[10]   NEURAL-GAS NETWORK FOR VECTOR QUANTIZATION AND ITS APPLICATION TO TIME-SERIES PREDICTION [J].
MARTINETZ, TM ;
BERKOVICH, SG ;
SCHULTEN, KJ .
IEEE TRANSACTIONS ON NEURAL NETWORKS, 1993, 4 (04) :558-569