A new feedforward neural network structural learning algorithm - Augmentation by training with residuals

A fully automatic feedforward neural network structural and weight learning algorithm is described. The Augmentation by Training with Residuals, ATR, requires neither guess of initial weight values nor the number of neurons in the hidden layer from users. The algorithm takes an incremental approach in which a hidden neuron is trained to model the mapping between the input and output of current exemplars, and is augmented to the existing network. The exemplars are then made orthogonal to the newly identified hidden neuron and used for the training of next hidden neuron. The improvement continues until a desired accuracy is reached This new structural and weight learning algorithm is applied to the identification of a two-degree-of-freedom planar robot, a Van der Pol oscillator and a Mackay-Glass equation. The algorithm is shown to be effective in modeling all three systems and is far superior to a linear modeling scheme in the case of the robot.