IMPROVEMENT OF THE BACKPROPAGATION ALGORITHM FOR TRAINING NEURAL NETWORKS

The application of artificial neural networks (ANNs) to chemical engineering problems, notably malfunction diagnosis, has recently been discussed (Hoskins and Himmelblau, Comput. chem. Engng 12, 881-890, 1988). ANNs "learn", from examples, a certain set of input-output mappings by optimizing weights on the branches that link the nodes of the ANN. Once the structure of the input-output space is learned, novel input patterns can be classified. The backpropagation (BP) algorithm using the generalized delta rule (GDR) for gradient calculation (Werbos, Ph.D. Thesis, Harvard University, 1974), has been popularized as a method of training ANNs. This method has the advantage of being readily adaptable to highly parallel hardware architectures. However, most current studies of ANNs are conducted primarily on serial rather than parallel processing machines. On serial machines, backpropagation is very inefficient and converges poorly. Some simple improvements, however, can render the algorithm much more robust and efficient. © 1990.