Selective multiple kernel learning for classification with ensemble strategy

Multiple Kernel Learning (MKL) aims to seek a better result than single kernel learning by combining a compact set of sub-kernels. However, MKL with L1-norm easily discards the sub-kernels with complementary information and MKL with Lp - norm ( p ≥ 2 ) often gets the redundant solution. To address these problems, a Selective Multiple Kernel Learning (SMKL) method, inspired by Ensemble Learning (EL), is proposed. Comparing MKL with Lp - norm ( p ≥ 2 ) , SMKL obtains a sparse solution by a pre-selection procedure. Comparing MKL with L1-norm, SMKL preserves the sub-kernels with complementary information by guaranteeing the high discrimination and large diversity of pre-selected sub-kernels. For quantifying the discrimination and diversity of sub-kernels, a new kernel evaluation is designed. SMKL reduces the scale of MKL optimization and saves the memory storing of the sub-kernels, which extends the scale of problem that MKL could solve. Specially, a fast SMKL method using L ∞ - norm constraint is focused, which needs no MKL optimization process. It means that the memory is hardly a limitation for MKL with the large scale problem. Experiments state that our method is effective for classification.