ANALOG VLSI NEURAL NETWORKS - IMPLEMENTATION ISSUES AND EXAMPLES IN OPTIMIZATION AND SUPERVISED LEARNING

Many time-critical neural network applications require fully parallel hardware implementations for maximal throughput. We first survey the rich array of technologies that are being pursued, then focus on the analog CMOS VLSI medium. Although analog VLSI holds great promise for implementing dense neural networks in a fully parallel manner, the medium is "messy" in that limited dynamic range, offset voltages, and noise sources all conspire to reduce precision. Many traditional neural models may be difficult to implement in analog technology. In this paper, we examine how neural networks may be directly implemented in analog VLSI, giving examples of approaches that have been pursued to date. Two important application areas are highlighted: Optimization, because neural hardware may offer a speed advantage of orders of magnitude over other methods, and supervised learning, because of the widespread use and generality of gradient-descent learning algorithms as applied to feedforward networks.