Chinese Text Classification Using Key Characters String Kernel

Most Chinese text classification methods are based on Chinese word segmentation and bag of words (BOW). The classification performance largely relies on the accuracy of segmentation. Unfortunately, perfect precision and disambiguation of segmentation cannot be reached. In order to solve this problem, a novel Chinese text classification method using string kernel is presented. String kernel computes the similarity of a pair of documents by comparing common substrings they have. Experimental results show that our method greatly enhances the classification on small training data sets. Although the performance of traditional string kernel is comparable to that of BOW methods on larger data set, the dimension of feature space is so high that the calculation process is very time-consuming. Our proposed key characters string kernel technique solves the efficiency and effectiveness problems. Experiments on larger data set show that SVM with key characters string kernel can achieve superior performance.