Convergence of gradient method with momentum for back-propagation neural networks

被引：17

作者：

Wu, Wei ^{[1
]}

Zhang, Naimin ^{[2
]}

Li, Zhengxue ^{[1
]}

Li, Long ^{[1
]}

Liu, Yan ^{[3
]}

机构：

[1] Dalian Univ Technol, Dept Appl Math, Dalian 116024, Peoples R China

[2] Wenzhou Univ, Math & Informat Sci Coll, Wenzhou 325035, Peoples R China

[3] Dalian Inst Light Ind, Coll Informat Sci & Engn, Dalian 116034, Peoples R China

来源：

JOURNAL OF COMPUTATIONAL MATHEMATICS | 2008年 / 26卷 / 04期

关键词：

back-propagation (BP) neural networks; gradient method; momentum; convergence;

D O I：

暂无

中图分类号：

O29 [应用数学];

学科分类号：

070104 [应用数学];

摘要：

In this work, a gradient method with momentum for BP neural networks is considered. The momentum coefficient is chosen in an adaptive manner to accelerate and stabilize the learning procedure of the network weights. Corresponding convergence results are proved.

引用

收藏

页码：613 / 623

页数：11

相关论文

共 14 条

[1]

Two highly efficient second-order algorithms for training feedforward networks [J].

Ampazis, N ;

Perantonis, SJ .

IEEE TRANSACTIONS ON NEURAL NETWORKS, 2002, 13 (05) :1064-1074

[2]

Steepest descent with momentum for quadratic functions is a version of the conjugate gradient method [J].

Bhaya, A ;

Kaszkurewicz, E .

NEURAL NETWORKS, 2004, 17 (01) :65-71

[3]

Fahlman S.E., 1988, Proceedings of the 1988 Connectionist Models Summer School, P38

[4]

Parameter convergence and learning curves for neural networks [J].

Fine, TL ;

Mukherjee, S .

NEURAL COMPUTATION, 1999, 11 (03) :747-769

[5]

DIFFUSION APPROXIMATIONS FOR THE CONSTANT LEARNING RATE BACKPROPAGATION ALGORITHM AND RESISTANCE TO LOCAL MINIMA [J].

FINNOFF, W .

NEURAL COMPUTATION, 1994, 6 (02) :285-295

[6]

Optimal convergence of on-line backpropagation [J].

Gori, M ;

Maggini, M .

IEEE TRANSACTIONS ON NEURAL NETWORKS, 1996, 7 (01) :251-254

[7]

ANALYSIS OF HIDDEN UNITS IN A LAYERED NETWORK TRAINED TO CLASSIFY SONAR TARGETS [J].

GORMAN, RP ;

SEJNOWSKI, TJ .

NEURAL NETWORKS, 1988, 1 (01) :75-89

[8]

ANALYSIS OF THE BACKPROPAGATION ALGORITHM WITH MOMENTUM [J].

PHANSALKAR, VV ;

SASTRY, PS .

IEEE TRANSACTIONS ON NEURAL NETWORKS, 1994, 5 (03) :505-506

[9]

Rumelhart D.E, 1986, EXPLORATIONS MICROST, V1

[10]

LEARNING REPRESENTATIONS BY BACK-PROPAGATING ERRORS [J].

RUMELHART, DE ;

HINTON, GE ;

WILLIAMS, RJ .

NATURE, 1986, 323 (6088) :533-536