Improved Comprehensive Measurement Feature Selection Method for Text Categorization

Text categorization plays an important role in applications where information is filtered, monitored, personalized, categorized, organized or searched. Feature selection remains as an effective and efficient technique in text categorization. Traditional feature selections ignored the effects of unbalanced categories and the distribution of a term in different categories.On this basis, we improved the Comprehensively Measure Feature Selection method (CMFS), and introduced the factors of category size and term distribution. The proposed method was compared and analyzed on Reuters 21,578 dataset using F1 measurement. Experimental results revealed that the proposed method performs better than five typical feature selections when SVM and NB classifiers are used.