Selective multiple kernel learning for classification with ensemble strategy

Multiple Kernel Learning (MKL) aims to seek a better result than single kernel learning by combining a compact set of sub-kernels. However, MKL. with L1-norm easily discards the sub-kernels with complementary information and MKL with Lp-norm(p >= 2) often gets the redundant solution. To address these problems, a Selective Multiple Kernel Learning (SMKL) method, inspired by Ensemble Learning (EL), is proposed. Comparing MKL with Lp-norm(p >= 2), SMKL obtains a sparse solution by a pre-selection procedure. Comparing MKL with Lp-norm, SMKL preserves the sub-kernels with complementary information by guaranteeing the high discrimination and large diversity of pre-selected sub-kernels. For quantifying the discrimination and diversity of sub-kernels, a new kernel evaluation is designed. SMKL reduces the scale of MKL optimization and saves the memory storing of the sub-kernels, which extends the scale of problem that MKL could solve. Specially, a fast SMKL method using L infinity-norm constraint is focused, which needs no MIC optimization process. It means that the memory is hardly a limitation for MKL with the large scale problem. Experiments state that our method is effective for classification. (C) 2013 Elsevier Ltd. All rights reserved.