Multinomial Least Angle Regression

Keerthi and Shevade (2007) proposed an efficient algorithm for constructing an approximate least angle regression least absolute shrinkage and selection operator solution path for logistic regression as a function of the regularization parameter. In this brief, their approach is extended to multinomial regression. We show that a brute-force approach leads to a multivariate approximation problem resulting in an infeasible path tracking algorithm. Instead, we introduce a noncanonical link function thereby: 1) repeatedly reusing the univariate approximation of Keerthi and Shevade, and 2) producing an optimization objective with a block-diagonal Hessian. We carry out an empirical study that shows the computational efficiency of the proposed technique. A MATLAB implementation is available from the author upon request.