A modified back-propagation method to avoid false local minima

被引:50
作者
Fukuoka, Y [1 ]
Matsuki, H
Minamitani, H
Ishida, A
机构
[1] Tokyo Med & Dent Univ, Inst Med & Dent Engn, Chiyoda Ku, Tokyo 1010062, Japan
[2] Keio Univ, Fac Sci & Technol, Kanagawa, Japan
关键词
back-propagation; false local minima; premature saturation; sigmoid derivative; weight readjusting; annealing;
D O I
10.1016/S0893-6080(98)00087-2
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
The back-propagation method encounters two problems in practice, i.e., slow learning progress and convergence to a false local minimum. The present study addresses the latter problem and proposes a modified back-propagation method. The basic idea of the method is to keep the sigmoid derivative relatively large while some of the error signals are large. For this purpose, each connecting weight in a network is multiplied by a factor in the range of (0,1), at a constant interval during a learning process. Results of numerical experiments substantiate the validity of the method. (C) 1998 Elsevier Science Ltd. All rights reserved.
引用
收藏
页码:1059 / 1072
页数:14
相关论文
共 31 条
[1]   MEAN FIELD ANNEALING - A FORMALISM FOR CONSTRUCTING GNC-LIKE ALGORITHMS [J].
BILBRO, GL ;
SNYDER, WE ;
GARNIER, SJ ;
GAULT, JW .
IEEE TRANSACTIONS ON NEURAL NETWORKS, 1992, 3 (01) :131-138
[2]  
Cybenko G., 1989, Mathematics of Control, Signals, and Systems, V2, P303, DOI 10.1007/BF02551274
[3]   ON THE APPROXIMATE REALIZATION OF CONTINUOUS-MAPPINGS BY NEURAL NETWORKS [J].
FUNAHASHI, K .
NEURAL NETWORKS, 1989, 2 (03) :183-192
[4]   STOCHASTIC RELAXATION, GIBBS DISTRIBUTIONS, AND THE BAYESIAN RESTORATION OF IMAGES [J].
GEMAN, S ;
GEMAN, D .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 1984, 6 (06) :721-741
[5]   MULTILAYER FEEDFORWARD NETWORKS ARE UNIVERSAL APPROXIMATORS [J].
HORNIK, K ;
STINCHCOMBE, M ;
WHITE, H .
NEURAL NETWORKS, 1989, 2 (05) :359-366
[6]   Progress in supervised neural networks [J].
Hush, Don R. ;
Horne, Bill G. .
IEEE SIGNAL PROCESSING MAGAZINE, 1993, 10 (01) :8-39
[7]   ERROR SURFACES FOR MULTILAYER PERCEPTRONS [J].
HUSH, DR ;
HORNE, B ;
SALAS, JM .
IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS, 1992, 22 (05) :1152-1161
[8]  
HUSH DR, 1988, P INT C NEURAL NETWO, V1, P441
[9]  
Irie B., 1988, IEEE INT C NEURAL NE, V1, P641
[10]   INCREASED RATES OF CONVERGENCE THROUGH LEARNING RATE ADAPTATION [J].
JACOBS, RA .
NEURAL NETWORKS, 1988, 1 (04) :295-307