Second-order methods for L1 regularized problems in machine learning

This paper proposes a Hessian-free Newton method for solving large-scale convex functions with an L1 regularization term. These problems arise in supervised machine learning models in which it is important to seek a sparse parameter vector. The proposed method operates in a batch setting, which is well suited for parallel computing environments, and employs sub-sampled Hessian information to accelerate progress of the iteration. The method consists of two phases, an active-set prediction phase that employs first-order and second-order information, and subspace phase that performs a Newton-like step. Numerical results on a speech recognition problem illustrate the practical behavior of the method.