A neural network training algorithm utilizing multiple sets of linear equations

A fast algorithm is presented for the training of multilayer perceptron neural networks, which uses separate error functions for each hidden unit and solves multiple sets of linear equations. The algorithm builds upon two previously described techniques. In each training iteration, output weight optimization (OWO) solves linear equations to optimize output weights, which are those connecting to output layer net functions. The method of hidden weight optimization (HWO) develops desired hidden unit net signals from delta functions. The resulting hidden unit error functions are minimized with respect to hidden weights, which are those feeding into hidden unit net functions. An algorithm is described for calculating the learning factor for hidden weights. We show that the combined technique, OWO-HWO is superior in terms of convergence to standard OWO-BP (output weight optimization-backpropagation) which uses OWO to update output weights and backpropagation to update hidden weights. We also show that the OWO-HWO algorithm usually converges to about the same training error as the Levenberg-Marquardt algorithm in an order of magnitude less time. (C) 1999 Published by Elsevier Science B.V. All rights reserved.