Blockwise projection matrix versus blockwise data on undersampled problems: Analysis, comparison and applications

Linear subspace methods are extensively used in many areas such as pattern recognition and machine learning. Among them, block subspace methods are efficient in terms of the computational complexity. In this paper, we perform a thorough analysis on block subspace methods and give a theoretical framework for understanding block subspace methods. It reveals the relationship between block subspace methods and classical subspace methods. We theoretically show that blockwise PCA has larger reconstruction errors than classical PCA and classical LDA has stronger discriminant power than blockwise LDA in the case of the same number of reduced features. In addition, based on the Fisher criterion, we also give a strategy for selecting an approximate block size for classification problems. The comprehensive experiments on face images and gene expression data are used to evaluate our results and a comparative analysis for various methods is made. Experimental results demonstrate that overly combining subspaces of block subspace methods without considering the subspace distance may yield undesirable performance on undersampled problems.