Using Semi-discrete Decomposition for Topic Identification

In the area of information retrieval, the dimension of document vectors plays an important role. We may need to find a few words or concepts, which characterize the document based on its contents, to overcome the problem of the "curse of dimensionality", which makes indexing of high-dimensional data problematic. To do so, we earlier proposed a Wordnet and Wordnet+LSI (latent semantic indexing) based model for dimension reduction. While LSI concepts contain identifiable terms in top-level concepts, we show in this paper that semi-discrete decomposition provides mostly smaller list of terms and we need to cope only with ternary weights. With this size of term list, the identification of document\´s topic becomes much more feasible.