Regularized orthogonal forward feature selection for spectral data

Feature selection for spectral data can be highly beneficial both to improve the predictive ability of the model and to greatly enhance its interpretation. This paper presents an efficient approach based on regularized orthogonal forward selection. The selection procedure is a direct optimization of model generalization capability by sequentially minimizing the leave-one-out (LOO) test error. Moreover, a regularization method is incorporated in order to further enforce model sparsity and generalization capability. The introduced algorithm is computationally very efficient, yet obtains a good feature subset that ensures the model generalization and interpretation. Comparisons with some of the existing state-of-art feature selection methods on several real data sets show that our algorithm performs fairly well with respect to computational efficiency and predict accuracy.