Sparse Kernels for Bayes Optimal Discriminant Analysis

Discriminant Analysis (DA) methods have demonstrated their utility in countless applications in computer vision and other areas of research - especially in the C class classification problem. The most popular approach is linear DA (LDA), which provides the C - 1-dimensional Bayes optimal solution, but only when all the class covariance matrices are identical. This is rarely the case in practice. To alleviate this restriction, Kernel LDA (KLDA) has been proposed. In this approach, we first (intrinsically) map the original nonlinear problem to a linear one and then use LDA to find the C - 1-dimensional Bayes optimal subspace. However, the use of KLDA is hampered by its computational cost, given by the number of training samples available and by the limitedness of LDA in providing a C - 1-dimensional solution space. In this paper, we first extend the definition of LDA to provide subspace of q < C - 1 dimensions where the Bayes error is minimized. Then, to reduce the computational burden of the derived solution, we define a sparse kernel representation, which is able to automatically select the most appropriate sample feature vectors that represent the kernel. We demonstrate the superiority of the proposed approach on several standard datasets. Comparisons are drawn with a large number of known DA algorithms.