Exploiting lexical information for function tag labeling

This paper proposes an novel approach to annotate function tags for unparsed text. What distinguishes our work from other attempts in such task is that we assign function tags directly basing on lexical information other than on parsed trees. In order to demonstrate the effectiveness and versatility of our method, we investigate two statistical models for automatic annotation, one is log-linear maximum entropy model and the other is margin maximum based support vector machine model, which achieve the best F-score of 82.8 and 86.4 respectively when tested on the text from Penn Chinese Treebank. We also quantity the effect of POS tagger accuracy on system performance. Our results indicate that the function tag types could be determined via flexible and powerful feature representations from words, POS tags and word position indicators, and that, similarly to syntactic parsing, the main difficulty lies in complex constituents with long-distance dependency.