Application of Query Sensitive Similarity Measure in IR systems

Document clustering has been widely used in information retrieval systems in order to improve the efficiency and also the effectiveness of ranked output systems using cluster hypothesis. This hypothesis states that relevant documents tend to be more similar to each other than to non-relevant documents, and therefore tend to appear in the same clusters. So far, the effectiveness of cluster hypothesis experimentally has been examined only for static-clustering and query-specific clustering using cosine similarity measure. On the other hand, the effectiveness of document clustering using query-sensitive similarity measure (QSSM) has been studied only with N-nearest neighbor test for very small and topic-specific document collections. In this paper, the cluster hypothesis for query-specific clustering is investigated using a query-sensitive similarity measure and a large document collection in an experimental environment. The results show that the cluster hypothesis holds for query-specific clustering using employed QSSM. And, the effectiveness of query-specific clustering will increase through the use of that QSSM.