Discriminant analysis: a unified approach

Linear discriminant analysis (LDA) as a dimension reduction method is widely used in data mining and machine learning. It however suffers from the small sample size (SSS) problem when data dimensionality is greater than the sample size. Many modified methods have been proposed to address some aspect of this difficulty from a particular viewpoint. A comprehensive framework that provides a complete solution to the SSS problem is still missing. In this paper, we provide a unified approach to LDA, and investigate the SSS problem in the framework of statistical learning theory. In such a unified approach, our analysis results in a deeper understanding of LDA. We demonstrate that LDA (and its nonlinear extension) belongs to the same framework where powerful classifiers such as support vector machines (SVMs) are formulated. In addition, this approach allows us to establish an error bound for LDA. Finally our experiments validate our theoretical analysis results.