Analysis on Effect Range of Context in Chinese Word Segmentation Based Word-Position Tagging

Chinese word segmentation (CWS) can be transformed into word-position-based approaches by conditional random field (CRF). It improved the performance of Chinese word segmentation greatly which makes it in wide use recently. When training on corpus with CRF, the size of feature window is the key to the training effect. To analyze the effect range of context, string sequence tagging segmentations are performed on Bakeoff2005 with toolkit CRF++0.53 and the results are: (1) contribution of below is greater than above, (2) size of feature window influencing the segment performance is no larger than 5, the proper size is four or five.