Sparse Kernel-Based Ensemble Learning With Fully Optimized Kernel Parameters for Hyperspectral Classification Problems

Recently, a kernel-based ensemble learning technique for hyperspectral detection/classification problems has been introduced by the authors, to provide robust classification over hyperspectral data with relatively high level of noise and background clutter. The kernel-based ensemble technique first randomly selects spectral feature subspaces from the input data. Each individual classifier, which is in fact a support vector machine (SVM), then independently conducts its own learning within its corresponding spectral feature subspace and hence constitutes a weak classifier. The decisions from these weak classifiers are equally or adaptively combined to generate the final ensemble decision. However, in such ensemble learning, little attempt has been previously made to jointly optimize the weak classifiers and the aggregating process for combining the subdecisions. The main goal of this paper is to achieve an optimal sparse combination of the subdecisions by jointly optimizing the separating hyperplane obtained by optimally combining the kernel matrices of the SVM classifiers and the corresponding weights of the subdecisions required for the aggregation process. Sparsity is induced by applying an l1 norm constraint on the weighting coefficients. Consequently, the weights of most of the subclassifiers become zero after the optimization, and only a few of the subclassifiers with non-zero weights contribute to the final ensemble decision. Moreover, in this paper, an algorithm to determine the optimal full-diagonal bandwidth parameters of the Gaussian kernels of the individual SVMs is also presented by minimizing the radius-margin bound. The optimized full-diagonal bandwidth Gaussian kernels are used by the sparse SVM ensemble to perform binary classification. The performance of the proposed technique with optimized kernel parameters is compared to that of the one with single-bandwidth parameter obtained using cross-validation by testing them on various data sets. On an average, the proposed sparse kernel-based ensemble learning algorithm with optimized full-diagonal bandwidth parameters shows an improvement of 20% over the existing ensemble learning techniques.