Maximum weight and minimum redundancy: A novel framework for feature subset selection

Feature subset selection is often required as a preliminary work for many pattern recognition problems. In this paper, a novel filter framework is presented to select optimal feature subset based on a maximum weight and minimum redundancy (MWMR) criterion. Since the weight of each feature indicates its importance for some ad hoc tasks (such as clustering and classification) and the redundancy represents the correlations among features. Through the proposed MWMR, we can select the feature subset in which the features are most beneficial to the subsequent tasks while the redundancy among them is minimal. Moreover, a pair-wise updating based iterative algorithm is introduced to solve our framework effectively. In the experiments, three feature weighting algorithms (Laplacian score, Fisher score and Constraint score) are combined with two redundancy measurement methods (Pearson correlation coefficient and Mutual information) to test the performances of proposed MWMR. The experimental results on five different databases (CMU PIE, Extended YaleB, Colon, DLBCL and PCMAC) demonstrate the advantage and efficiency of our MWMR.