Special-purpose digital hardware for neural networks: An architectural survey

This paper presents a survey of digital systems to implement neural networks. We consider two basic options for designing these systems: parallel systems with standard digital components and parallel systems with custom processors. We describe many examples under each option, with an emphasis on commercially available systems. We report a first trend toward more general architectures and a second trend toward simple and fast structures. We discuss our experience in running a small ANN problem on two of these machines. After a reasonable programming effort, we obtain good convergence, but most of the training times are actually slower or moderately faster than on a serial workstation. We conclude that it is important to chose one's problems carefully, and that support software and in general, system integration, is only beginning to reach the level of versatility that many researchers will require.