Pattern classification with ordered features using mRMR and neural networks

Feature selection and feature extraction are widely used feature reduction approaches which are insufficient for coping with high-dimensional pattern classification problems when all features of the problem have the same significance. A proved useful method for solving this problem is incremental attribute learning (IAL) which gradually trains input features in one or more size. Hence a new preprocessing called feature ordering should be introduced in pattern classification. In previous IAL studies of pattern classification, feature ordering was ranked by discrimination ability which was derived from a neural network with a single input and all outputs. Such a method which is similar to wrappers in feature selection is applicable for high-dimensional pattern classification problems. However, due to the fact that it is time-consuming, a substitute approach for feature ordering is presented in this paper, where feature ordering is ranked by redundancy and relevance using mRMR method. Experimental results show that feature ordering derived by mRMR can not only save time, but also obtain the best classification rate based on ITID, a neural IAL model, compared with those in previous studies.