ENERGY FUNCTIONS FOR MINIMIZING MISCLASSIFICATION ERROR WITH MINIMUM-COMPLEXITY NETWORKS

For automatic target recognition, a neural network is desired that minimizes the number of misclassifications with the minimum network complexity. Minimizing network complexity is important for both improving generalization and simplifying implementation. The least mean squares (LMS) energy function used in standard back propagation does not always produce such a network. Therefore, two minimum misclassification error (MME) energy functions are advanced to achieve this. Examples are given in which LMS requires five times as many hidden units in a multilayer perceptron to achieve test set classification accuracy similar to that achieved with the MME functions. Examples are given to provide insight into the nature of the LMS performance, namely that LMS approximates the a posteriori probabilities and class boundaries emerge indirectly from this process. The examples also show that the MME functions tend to find local minima less often than LMS does for the same number of hidden units. This is believed to be due to the difference in network complexity needed to accurately approximate a posteriori probabilities versus class boundaries.