Maximum Entropy Model Based Classification with Feature Selection

In this paper, we propose a classification algorithm based on the maximum entropy principle. This algorithm finds the most appropriate class-conditional maximum entropy distributions for classification. No prior knowledge about the form of density function for estimating the class conditional density is assumed except that the information is given in the form of expected valued of features. This algorithm also incorporates a method to select relevant features for classification. The proposed algorithm is suitable for large data-sets and is demonstrated by simulation results on some real world benchmark data-sets.