A Novel Multilingual Text Categorization System using Latent Semantic Indexing

Latent semantic indexing is a well known technique in information retrieval, especially in dealing with polysemy and synonymy. LSI use SVD process to decompose the original term-document matrix into a lower dimension triplet. The triplet (the resulted matrices) is the approximation to original matrix and can capture the latent semantic relation between terms. In this paper, we propose a novel method for multilingual text categorization using latent semantic indexing. The centroid of each class has been calculated in the decomposed SVD space. The similarity threshold of categorization is predefined for each centroid. Test documents with similarity measurement larger than the threshold is labeled "positive" (relevant) or else would be labeled "negative" (non-relevant). Experimental result indicated that the performance on the precision, recall and F1 are quite good using LSI technique to categorize the multi-language text. The F1 measurement has an average value of 70% and the precision can reach 80% using our algorithm