REDUCTION OF REQUIRED PRECISION BITS FOR BACKPROPAGATION APPLIED TO PATTERN-RECOGNITION

The number of precision bits for operations and data are limited in the hardware implementations of back-propagation (BP). Reduction of rounding error due to this limited precision is crucial in the implementation. Our new learning algorithm is based on overestimation of significant error in order to alleviate underflow and omission of weight updating for correctly recognized patterns. While the conventional BP algorithm minimizes the squared error between output signals and supervising data, our learning algorithm minimizes the Weighted Error Function. In the learning simulation of multifont capital recognition, our algorithm converged recognition accuracy to 100% with only 8-bit precision. In addition, the recognition accuracy for characters that did not appear in the training data reached 94.9%. This performance is equivalent to that of a conventional BP with 12-bit precision. Moreover, we found the performance of the Weighted Error Function to be high even when only a small number of hidden neurons were used. Consequently, our algorithm reduces the required amount of weight memory.