L1 regularized projection pursuit for additive model learning

In this paper, we present a L₁ regularized projection pursuit algorithm for additive model learning. Two new algorithms are developed for regression and classification respectively: sparse projection pursuit regression and sparse Jensen-Shannon Boosting. The introduced L₁ regularized projection pursuit encourages sparse solutions, thus our new algorithms are robust to overfitting and present better generalization ability especially in settings with many irrelevant input features and noisy data. To make the optimization with L₁ regularization more efficient, we develop an ldquoinformative feature firstrdquo sequential optimization algorithm. Extensive experiments demonstrate the effectiveness of our proposed approach.