Accurate Chinese Text Classification via Multiple Strategies

Text classification is one of means to understand text content. It is widely used in information retrieving, filtering spam, monitoring ill gossips, and blocking pornographic and evil messages. kN N is widely used in text categorization, but it suffers from biased training data set. In developing Prototype of Internet Information Security for Shanghai Council of Information and Security, we detect that when training data set is biased, almost all test documents of some rare (smaller) categories are classified into common (larger) ones by traditional kN N classifier. The performance of text classification can not satisfy the user´s requirement in this case. To alleviate such a misfortune, we adopt 2 measures to boost kN N classifier. Firstly, we optimize features by removing some candidate features. Secondly, we modify traditional decision rules by integrating number of training samples of each category with them. Exhaustive experiments illustrate that the adapted kN N achieves significant classification performance improvement on biased corpora.