An Empirical Evaluation of Supervised Dimensionality Reduction for Recognition

In the literature, many dimensionality reduction methods have been proposed and applied to recognition tasks, including handwritten digits recognition, character recognition and string recognition. However, it is usually difficult for the researchers to decide which method is the optimal choice for the problem at hand. In this paper, we empirically compare some supervised dimensionality reduction methods on handwritten digits recognition, English letter recognition and ancient Arabic sub word recognition, to evaluate their performance on the recognition tasks. These compared methods include traditional linear dimensionality reduction approach (linear discriminant analysis, LDA), locality-based manifold learning approach (marginal Fisher analysis, MFA) and relational learning approach (probabilistic relational principal component analysis, PRPCA). Experimental results and statistical tests show that locality-based manifold learning approach (MFA) generally performs well in terms of recognition accuracy, but with high computational complexity, traditional linear dimensionality reduction approach (LDA) is efficient, but not necessarily to deliver the best result, relational learning approach (PRPCA) is promising, and more efforts should be dedicated to this area.