On the impact of fusion strategies on classification errors for large ensembles of classifiers

The growing availability of sensor networks brings practical situations where a large number of classifiers can be used for building a classifier ensemble. In the most general case involving sensor networks, the classifiers are fed with multiple inputs collected at different locations. However, classifier fusion is often studied within an idealized formulation where each classifier is fed with the same point in the feature space, and estimate the posterior class probability given this input. We first expand this formulation to situations where classifiers are fed with multiple inputs, demonstrating the relevance of the formulation to situations involving sensor networks, and a large number of classifiers. Following that, we determine the rate of convergence of the classification error of a classifier ensemble for three fusion strategies (average, median and maximum) when the number of classifiers becomes large. As the size of the ensemble increases, the best strategy is defined as the one that results in fastest convergence of the classification error to zero. The best strategy is analytically shown to depend on the distribution of the individual classification errors: average is the best for normal distributions; maximum is the best for uniform distributions; and median is the best for Cauchy distributions. The general effect of heavy-tailedness is also analytically investigated for the average and median strategies. The median strategy is shown to be robust to heavy-tailedness, while performance of the average strategy is shown to degrade as heavy-tailedness becomes more pronounced. The combined effects of bimodality and heavy-tailedness are also investigated when the number of classifiers become large. (c) 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.