An improved term weighting scheme for vector space model

Document representation has been the fundamental issue in the information retrieval (IR). However, the traditional vector space model (VSM) has data sparseness phenomena on the representation of document vectors, and cannot well discriminate the expression competence to the document content of indexing terms in different positions. This paper proposes an improved term weighting method by introducing information gain of terms while taking above factors into account. The theoretical analysis and experimental results show that the new scheme improves the performance of VSM in IR in terms of higher recall and precision.