Lp-regularized optimization by using orthant-wise approach for inducing sparsity

Sparsity induced in the optimized weights effectively works for factorization with robustness to noises and for classification with feature selection. For enhancing the sparsity, L₁ regularization is introduced into the objective cost function to be minimized. In general, however, L_p (p<;1) regularization leads to more sparse solutions than L₁, though L_p regularized problem is difficult to be effectively optimized. In this paper, we propose a method to efficiently optimize the L_p regularized problem. The method reduces the L_p problem into L₁ regularized one via transforming target variables by the mapping based on L_p, and optimizes it by using orthant-wise approach. In the proposed method, the L_p problem is directly optimized for computational efficiency without reformulating it into iteratively reweighting scheme. The proposed method is generally applicable to various problems with L_p regularization, such as factorization and classification. In the experiments on the classification using logistic regression and factorization based on least squares, the proposed method produces favorable sparse results.