Chinese POS tagging based on bilexical co-occurrences

Chinese part of speech tagging is the basis of Chinese information processing. This paper proposes a method based on bilexical co-occurrences to tag Chinese text. The standard hidden Markov model assumes the transition between states (part of speech) is independent of the observation (word) sequence and the generation of a new observation is independent of other observations. In fact, Chinese text does not satisfy this assumption. Based on hidden Markov model, the effect of the words in the context on the decision of part of speech is also considered. The discriminative ability of the model is improved. Deleted interpolation is utilized to mitigate the data sparseness problem. We evaluate the proposed model on PFR China Daily corpus. The tagging accuracy is 99.09% on closed test and 96.37% on open test.