Title :
Transductive HMM based Chinese text chunking
Author :
Li, Heng ; Webster, Jonathan J. ; Kit, Chunyu ; Yao, Tianshun
Author_Institution :
Inst. of Comput. Software & Theor., Northeastern Univ., Shenyang, China
Abstract :
We present a novel methodology to enhance Chinese text chunking with the aid of transductive Hidden Markov Models (transductive HMMs, henceforth). We consider chunking as a special tagging problem and attempt to utilize, via a number of transformation functions, as much relevant contextual information as possible for model training. These functions enable the models to make use of contextual information to a greater extent and keep us away from costly changes of the original training and tagging process. Each of them results in an individual model with certain pros and cons. Through a number of experiments, we succeed in integrating the best two models into a significantly better one. We carry out the chunking experiments on the HIT Chinese Treebank corpus. Experimental results show that it is an effective approach, achieving an F score of 82.38%.
Keywords :
hidden Markov models; natural languages; text analysis; Chinese text chunking; contextual information; model training; transductive Hidden Markov Model; transformation function; Context modeling; Entropy; Hidden Markov models; Learning systems; Machine learning; Natural language processing; Probability distribution; Software testing; Support vector machines; Tagging;
Conference_Titel :
Natural Language Processing and Knowledge Engineering, 2003. Proceedings. 2003 International Conference on
Conference_Location :
Beijing, China
Print_ISBN :
0-7803-7902-0
DOI :
10.1109/NLPKE.2003.1275909