In this paper we discuss some possible ways to combine the outputs of a set of neural network classifiers to reach a combined decision with a higher performance, in terms of lower rejection rates and/or better accuracy rates. The methods considered range from the requirement of a complete agreement among the individual classifications to election schemes based on the distribution of votes collected by the different classes. In addition, the rejection rules based on the different output classes can be complemented by rules that also consider the information in the individual output vectors, with the possibility of using threshold requirements and that of averaging the different vectors. Although the Bayesian framework and some probabilistic assumptions provide useful indications about the potential advantage of different combination schemes, the combined performance ultimately depends on the joint probability distribution of the outputs, and it can be estimated by joining the results of different nets on the same test set. The combination methods are very flexible, they permit a straightforward cooperation of neural and traditional recognizers, and they are appropriate in a development environment where experiments are performed with different kinds of nets and features for a selected application. From our experiments in the field of handwritten digit recognition (up to a total of more than 50,000 characters), we found that the use of a small number of nets (two to three) with a sufficiently large uncorrelation in their mistakes reaches a combined performance that is significantly higher than the best obtainable from the individual nets, with a negligible effort after starting from a pool of networks produced in the development phase of an application. In particular for a real-world OCR application, the best accuracy increase is about half the increase in the rejection rate, so that accuracies of the order of 99.5% can be reached by rejecting less than 5% of the patterns. This performance is significant for real applications.
