Comparative Advantage Approach for Sparse Text Data Clustering

Document clustering is the process of partitioning a set of unlabeled n documents into clusters such that documents in each cluster share some common concepts. Each concept is conveniently represented by some key terms. Using words as features, text data are represented as a vector in a very high dimensional vector space. However, most documents are sparse vectors, for example, more than ten thousand dimensions and sparsity of 98%. In this paper, we study a fast classification algorithm based on the idea of comparative advantage for clustering sparse data. The proposed algorithm uses one "ruler" instead of k centers to identify the comparative advantage of each cluster and define the cluster label for each document. Experimental results show that our algorithm has comparable performance but faster than k-means. It can produce clusters with smaller overlapping concepts in the sense of key terms.