Text classification in fragmented sublanguage domains

The unique problems involved in developing text classification systems for texts that have low conceptual predictability are addressed. The authors present a shell called the FLUE (fragmented language understanding environment), which is capable of generating applications in fragmented sublanguage domains. The FLUE combines an expressive concept representation with a robust parsing technique called piecewise parsing. A common source of classification failure is unrecognized lexemes. The representation of concepts leverages differences in word class restrictions in order to learn unknown lexemes. The parser´s parallel search for concepts also gives it a large measure of immunity to the conceptual unpredictabilities of fragmented texts. Yet, the technique can be scaled to accommodate more grammatical texts, something not usually possible for other systems. Although the authors demonstrate the technique on English language texts, it is applicable to texts in other languages