Bi-level dimensionality reduction methods using feature selection and feature extraction

Variety of feature selection methods have been developed in the literature, which can be classified into three main categories: filter, wrapper and hybrid approaches. Filter methods apply an independent test without involving any learning algorithm, while wrapper methods require a predetermined learning algorithm for feature subset evaluation. Filter and wrapper methods have their drawbacks and are complementary to each other. The filter approaches have low computational cost with insufficient reliability in classification while wrapper methods tend to have superior classification accuracy but require great computational effort. The methods proposed in this paper are bi-level dimensionality reduction methods that integrate filter method and feature extraction method with the aim to improve the classification performance of the features selected. In the two approaches proposed, in level 1 of dimensionality reduction, feature are selected based on mutual correlation and in level 2 selected features are used to extract features using PCA or LPP. To evaluate the performance of the proposed methods several experiments are conducted on standard datasets and the results obtained show superiority of the proposed methods over single level dimensionality reduction techniques (feature selection based on Mutual correlation, PCA and LPP).