Dynamics of multilayer networks in the vicinity of temporary minima

被引：24

作者：

Ampazis, N

Perantonis, SJ

Taylor, JG ^{[1
]}

机构：

[1] Univ London Kings Coll, Dept Math, London WC2R 2LS, England

[2] Natl Res Ctr Demokritos, Inst Informat & Telecommun, Athens, Greece

来源：

NEURAL NETWORKS | 1999年 / 12卷 / 01期

关键词：

feed-forward neural networks; supervised learning; back-propagation; temporary minima; dynamical systems; Jacobian matrix; eigenvalues; constrained optimization;

D O I：

10.1016/S0893-6080(98)00103-8

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 [模式识别与智能系统]; 0812 [计算机科学与技术]; 0835 [软件工程]; 1405 [智能科学与技术];

摘要：

A dynamical system model is derived for a single-output, two-layer neural network, which learns according to the back-propagation algorithm. Particular emphasis is placed on the analysis of the occurrence of temporary minima. The Jacobian matrix of the system is derived, whose eigenvalues characterize the evolution of learning. Temporary minima correspond to critical points of the phase plane trajectories, and the bifurcation of the Jacobian matrix eigenvalues signifies their abandonment. Following this analysis, we show that the employment of constrained optimization methods can decrease the time spent in the vicinity of this type of minima. A number of numerical results illustrates the analytical conclusions. (C) 1999 Elsevier Science Ltd. All rights reserved.

引用

页码：43 / 58

页数：16

共 25 条

[1]

LEARNING BY ONLINE GRADIENT DESCENT [J].