DocumentCode
2664800
Title
Transductive HMM based Chinese text chunking
Author
Li, Heng ; Webster, Jonathan J. ; Kit, Chunyu ; Yao, Tianshun
Author_Institution
Inst. of Comput. Software & Theor., Northeastern Univ., Shenyang, China
fYear
2003
fDate
26-29 Oct. 2003
Firstpage
257
Lastpage
262
Abstract
We present a novel methodology to enhance Chinese text chunking with the aid of transductive Hidden Markov Models (transductive HMMs, henceforth). We consider chunking as a special tagging problem and attempt to utilize, via a number of transformation functions, as much relevant contextual information as possible for model training. These functions enable the models to make use of contextual information to a greater extent and keep us away from costly changes of the original training and tagging process. Each of them results in an individual model with certain pros and cons. Through a number of experiments, we succeed in integrating the best two models into a significantly better one. We carry out the chunking experiments on the HIT Chinese Treebank corpus. Experimental results show that it is an effective approach, achieving an F score of 82.38%.
Keywords
hidden Markov models; natural languages; text analysis; Chinese text chunking; contextual information; model training; transductive Hidden Markov Model; transformation function; Context modeling; Entropy; Hidden Markov models; Learning systems; Machine learning; Natural language processing; Probability distribution; Software testing; Support vector machines; Tagging;
fLanguage
English
Publisher
ieee
Conference_Titel
Natural Language Processing and Knowledge Engineering, 2003. Proceedings. 2003 International Conference on
Conference_Location
Beijing, China
Print_ISBN
0-7803-7902-0
Type
conf
DOI
10.1109/NLPKE.2003.1275909
Filename
1275909
Link To Document