THE OFFSET ALGORITHM - BUILDING AND LEARNING-METHOD FOR MULTILAYER NEURAL NETWORKS

A general method for building and training a multilayer neural network functioning as a parity machine is proposed. It is composed of two basic steps: a growth step in which two hidden layers are built and a pruning step in which any redundant units are removed. A perceptron-type algorithm is used to learn the connection strengths in order to minimize a classification error. The first hidden layer is built by adding units as they are needed, until the zero error convergence is achieved. We then show that the problem of mapping these internal representations onto the desired output is the n-parity problem. So, the second hidden layer is built by a geometrical design procedure with no learning. The used pruning process can remove sometimes all the units of the second hidden layer. The final architecture can then have one or two hidden layers.