Extracting Relevance Information from User Click through Data Using Conditional Random Field

It is now widely recognized that user interactions with search results can provide substantial relevance information on the documents. In this paper, we focus on extracting relevance information from one source of user interactions, user click-through data which record the sequence of documents being clicked in the result sets during a user search session. We emphasize the importance of the temporal nature of user click patterns and use conditional random fields to model its relations with the degree of relevance of the individual documents in the result sets. A key advantage of our models and algorithms is their ability to express the long-distance inter-actions conditioned on the click patterns. We test our algorithms using click-through data from a commercial search engine and evaluate the extracted relevance grades against those assigned by human judges.