Distributed Feature Representations for Dependency Parsing

This paper presents an approach to automatically learning distributed representations for features to address the feature sparseness problem for dependency parsing. Borrowing terminologies from word embeddings, we call the feature representation feature embeddings. In our approach, the feature embeddings are inferred from large amounts of auto-parsed data. First, the sentences in raw data are parsed by a baseline system and we obtain dependency trees. Then, we represent each model feature using the surrounding features on the dependency trees. Based on the representation of surrounding context, we proposed two learning methods to infer feature embeddings. Finally, based on feature embeddings, we present a set of new features for graph-based dependency parsing models. The new parsers can not only make full use of well-established hand-designed features but also benefit from the hidden-class representations of features. Experiments on the standard Chinese and English data sets show that the new parser achieves significant performance improvements over a strong baseline.