Title :
An Improved Chinese Segmentation Algorithm Based on New Dictionary Construction
Author :
Niu, Yan ; Li, Lala
Author_Institution :
Comput. Coll., Hubei Univ. of Technol., Wuhan, China
Abstract :
In this paper, we make use of the result of word frequency statistics design a new dictionary construction and propose an improved FMM algorithm which based on analysis the principle and characteristics of traditional FMM algorithm. Through the time complexity analysis and experimental comparison, the improved FMM algorithm can further improve the efficiency of the Chinese word segmentation.
Keywords :
dictionaries; natural language processing; statistical analysis; Chinese segmentation algorithm; dictionary construction; forward maximum matching algorithm; word frequency statistics; Algorithm design and analysis; Design engineering; Dictionaries; Educational institutions; Frequency; Handicapped aids; Information processing; Natural languages; Statistical analysis; Vocabulary; Chinese Segmentation; Chinese word frequency; FMM algorithm; segmentation dictionary;
Conference_Titel :
Computational Science and Engineering, 2009. CSE '09. International Conference on
Conference_Location :
Vancouver, BC
Print_ISBN :
978-1-4244-5334-4
Electronic_ISBN :
978-0-7695-3823-5
DOI :
10.1109/CSE.2009.269
