Title :
Problems and Review of Statistical Parsing Language Model
Author :
Zhou, Faguo ; Zhang, Fan ; Yang, Bingru
Author_Institution :
Sch. of Mech. Electron. & Inf. Eng., China Univ. of Min. & Technol., Beijing, China
Abstract :
The lexical language model is recently the hotspot in grammar research, which is promoted by incorporating the phrase head with statistics. This paper summarizes about four improving language models which belong to this kind of model: they have utilized heads that is extracted by CFG and calculated the probability between the heads or inside CFG. Different from N-gram and SCFG, the probability calculation change SCFG´s manner to the calculation among vocabulary. Corresponding experiments have shown improvement not only in precision but also in the language complexity. This paper has analyzed these models and also pointed out their problems. Especially, more detailed discussion was given about the immediate-head language model. But, it still has many defects. Finally, the potential idea for Chinese natural language model was promoted, the feasibility and the actuality of Chinese deep grammar analysis have discussed, too.
Keywords :
grammars; natural language processing; probability; statistical analysis; vocabulary; grammar analysis; immediate head language model; language complexity; lexical language; n-gram; natural language model; probability calculation; statistical parsing language model; vocabulary; Analytical models; Computational modeling; Grammar; Mathematical model; Predictive models; Probability; Syntactics; head phrase structure grammar; long distance dependencies; syntax language model; the statistical parsing;
Conference_Titel :
Asian Language Processing (IALP), 2010 International Conference on
Conference_Location :
Harbin
Print_ISBN :
978-1-4244-9063-9
DOI :
10.1109/IALP.2010.51