Learning Continuous Word Representations from Large-Scale Corpus through Linear Approach

This paper aims to find a mathematical and statistical way to express natural words´ semantic information by mapping words onto a high-dimension continuous space. This paper presents a new approach of training word representations which 1) uses Weighted Local Influence language model (WLI model) by assigning different weight to the words on different locations, 2) introduces polynomial linear regression into the training of word representations, and 3) presents a global optimization scheme introduces polynomial linear regression into global optimal solution word representations. Empirical evidence indicates the availability and effectiveness of the distributed representations based on WLI model. A 7-dimensional sentiment lexicon is constructed based on word representations to prove WLI´s semantic descriptiveness and the feasibility of linear empirical functions. Finally this paper discusses over some possible improvements on WLI model in future work.