CAN BACKPROPAGATION ERROR SURFACE NOT HAVE LOCAL MINIMA

From a theoretic point of view, we show that for an arbitrary T-element training set with t(t less-than-or-equal-to T) different inputs, the backpropagation error surface does not have suboptimal local minima if the network is capable of exactly implementing an arbitrary training set consisting of t different patterns. As a special case, the error surface of a backpropagation network with one hidden layer and t - 1 hidden units has no local minima, if the network is trained by an arbitrary T-element set with t different inputs.