Bigram Chinese Word Segmentation by Viterbi Algorithm

Chinese word segmentation is an important foundation for Chinese information processing. This paper proposes a new Chinese word segmentation model based on Bayesian network. In this model, Character alignment Viterbi algorithm, which treats the preceding word of each Chinese character as its state, and the N-gram probability as its state transition probability, is suggested to be combined with Viterbi algorithm to achieve better performance. The model we proposed also achieves word sense disambiguation and auto recognition of foreign and domestic person names together. It is demonstrated to be more efficient in word segmentation under better precision and recall.