Orthogonalized discriminant analysis based on generalized singular value decomposition

Generalized singular value decomposition (GSVD) has been used for linear discriminant analysis (LDA) to solve the small sample size problem in pattern recognition. However, this algorithm may suffer from the over-fitting problem. In this paper, we propose a novel orthogonalization technique for the LDA/GSVD algorithm to address the over-fitting problem. In this technique, an orthogonalization of the basis of the discriminant subspace derived from the LDA/GSVD algorithm is carried out through an eigen-decomposition of a small size inner product matrix. It is computationally efficient when data are high dimensional. The technique is further applied to the kernelized LDA/GSVD algorithm, mGSVD-KDA, leading to a new algorithm, referred to as GSVD-OKDA. It is shown that with linear and nonlinear kernels, this new algorithm successfully overcomes the over-fitting problem of the LDA/GSVD and mGSVD-KDA algorithms. Simulation results show that the proposed algorithms provide high recognition accuracy with low computational complexity.