Learning to identify core term of knowledge unit from short text

We present a new task of identifying core term (CT) of knowledge unit (KU) from text for knowledge management and service. Two kinds of approaches, including binary classification using naïve bayesian, decision tree, logistic regression and SVM, as well as competition learning based on pairwise classification, are investigated for this specific task, combined with presented rich feature set from position, token features to statistic and linguistic features. Experimental results show that simple classification method can effectively address this task with desirable performance at 82.7% KU accuracy. However, since the recognition of core term relies on the KU as an integer and all its inner terms, competition learning based on pairwise classification achieves better result at 89.6%. We also empirically show that all of the presented types of features are useful for our task, and the combination of position and linguistic features is essential for information extraction on short text.