Title :
A new collocation extraction method combining multiple association measures
Author :
Lin, Jian-fang ; Li, Sheng ; Cai, Yuhan
Author_Institution :
MOE-MS Key Lab. of NLP & Speech, Harbin Inst. of Technol., Harbin
Abstract :
As an important linguistic resource, collocation represents a significant relation between words. Automatic collocation extraction is very important for many natural language processing applications, such as word sense disambiguation, machine translation and information retrieval etc. While traditional collocation extraction approaches use only one single statistical measure, they may not be optimal in that they can not take advantage of multiple statistical measures. In this paper, we propose a logistic linear regression model (LLRM) that combines five classical lexical association measures: x2-test, t-test, co-occurrence frequency, log-likelihood ratio and mutual information. Experiments show that our approach leads to a significant performance improvement in comparison with individual basic methods in both precision and recall.
Keywords :
natural language processing; regression analysis; text analysis; automatic collocation extraction; collocation extraction method; information retrieval; log-likelihood ratio; logistic linear regression model; machine translation; multiple association measures; natural language processing applications; Cybernetics; Data mining; Frequency measurement; Information retrieval; Laboratories; Linear regression; Logistics; Machine learning; Magnetic heads; Mutual information; Co-occurrence frequency; Collocation; Log-likelihood ratio; Mutual information; T-test; X2-test;
Conference_Titel :
Machine Learning and Cybernetics, 2008 International Conference on
Conference_Location :
Kunming
Print_ISBN :
978-1-4244-2095-7
Electronic_ISBN :
978-1-4244-2096-4
DOI :
10.1109/ICMLC.2008.4620370