Text Classificationg for Imbalanced Data Sets

Imbalanced data set has caused a significant drawback of the classification performance attainable by most normal machine learning algorithm. However, the samples are often imbalanced. Therefore, how to reduce the effects of uneven distribution of training sets on text classification performance is a great challenge for machine learning on imbalanced data sets. Currently, the study on imbalaced data mainly lies in two aspects: data-level and algorithm-level. The paper focuses on the study of the three solutions: sample set restructuring, enhancement method of feature selection and weight retouch. Experimental results show that these methods are effective in improving classification performance. After comparing and analyzing the effects of these methods based on the experiments, this paper gets expressly some useful conclusions for some key issues, such as which sampling texts should be chosen and how many sampling texts should be decided for sample restructuring, how about defining separate threshold for each category in feature selection and how to adjust the weights in classification algorithm.