Segmentation of Chinese word based on method of rough segment and part of speech tagging

Author

Jiang Fang ; Yue Xiang ; Li Guo-he ; Wu Weijiang

Author_Institution

Coll. of Geophys. & Inf. Eng., China Univ. of Pet., Beijing, China

fYear

2013

fDate

12-13 Oct. 2013

Firstpage

538

Lastpage

542

Abstract

The segmentation of Chinese words from text documents is one of important contents of Chinese information processing. After every segmentation of Chinese words is obtained by the Chinese word rough segmentation by maximum match and ambiguity detection algorithms, each word in every rough segmentation is tagged by Viterbi algorithm according to HMM model of part-of-speech tagging. At last, each rough segmentation is estimated by the definition of optimal estimation function of part-of-speech tagging, and then the best one is selected as the optimal segmentation. The segmentation presented is better than others by the comparison of experiments.

Keywords

classification; hidden Markov models; text analysis; word processing; Chinese word rough segmentation; HMM model; Viterbi algorithm; ambiguity detection algorithms; hidden Markov model; maximum match algorithms; optimal estimation function; part-of-speech tagging; Accuracy; Educational institutions; Hidden Markov models; Information processing; Speech; Tagging; Viterbi algorithm; HMM; Viterbi Algorithm; part-of-speech tagging; word segmentation;

fLanguage

English

Publisher

ieee

Conference_Titel

Computer Science and Network Technology (ICCSNT), 2013 3rd International Conference on

Conference_Location

Dalian

Type

conf

DOI

10.1109/ICCSNT.2013.6967171

Filename

6967171