Ensemble learning with kernel mapping

Kernel learning is an important learning framework in machine learning, whose main idea is a mapping from input space to feature space induced by kernel function which yields a linear separation problem in the feature space. However, the generalization ability of kernel learning, which may lead to over-fitting of training data, has not been formally taken into consideration in previous literatures. We propose to tackle this problem by adopting ensemble learning in feature space. By bootstrapping training data set, several slightly different sets are obtained, with which we build up several slightly different kernels. The generated kernels are plugged into decision tree based learners to conduct similarity based learning and finally we combine all learners with a majority voting strategy. The proposed algorithm is tested in the famous UCI data repository with comparison to some previous baseline algorithms to show its effectiveness.