SUPERSAB - FAST ADAPTIVE BACK PROPAGATION WITH GOOD SCALING PROPERTIES

This paper presents SuperSAB: an adaptive acceleration strategy for error back propagation learning. The strategy is compared with the original back propagation algorithm, as well as with previously proposed acceleration techniques. It will be shown that SuperSAB may converge orders of magnitude faster than the original back propagation algorithm, and is only slightly instable. In addition, the algorithm is very insensitive to the choice of parameter values, and has excellent scaling properties. All simulations have been carried out on the Edinburgh concurrent supercomputer: a large parallel computer system, based on transputers. The power of this machine made it possible to test both SuperSAB and the original back propagation algorithm very extensively. As a result, this paper presents some interesting phenomenology results on the influence of parameter values in the original back propagation algorithm. © 1990.