WEIGHT PERTURBATION - AN OPTIMAL ARCHITECTURE AND LEARNING TECHNIQUE FOR ANALOG VLSI FEEDFORWARD AND RECURRENT MULTILAYER NETWORKS

Previous work on analog VLSI implementation of multilayer perceptrons with on-chip learning has mainly targeted the implementation of algorithms such as back-propagation. Although back-propagation is efficient, its implementation in analog VLSI requires excessive computational hardware. In this paper we show that using gradient descent with direct approximation of the gradient instead of back-propagation is more economical for parallel analog implementations. We also show that this technique (which we call "weight perturbation") is suitable for multilayer recurrent networks as well. A discrete level analog implementation showing the training of an XOR network as an example is also presented.