Feature selection based on information theory for pattern classification

Feature selection acts as a significant problem for pattern classification systems. We discuss about how to select valuable features according to the maximal statistical dependency criterion based on mutual information. In majority of datasets the features are not independent and their combination delivers more vital information than their individual forecast. In this paper we propose a feature selection method for semi supervised classification based upon the influence of information theory which provides a reliable measure of relation between the classes and features. A hybrid feature selection method invoking information theory is proposed. The implementation is also validated with two freely available datasets acquired from UCI and NCI data repositories. The significance of the complete estimation of mutual information is discussed when employed as a feature selection criterion.