Learning relevance from click data via neural network based similarity models

We introduce a new neural network based similarity model for learning document relevance under a query. The main idea is to use the binomial distribution to model the proportion of people who clicked document d under query q among the users who viewed d under q. Our model is a generalization of existing neural network based latent semantic models in that both its objective function and its parametrization of the user click probability generalizes the existing ones. Compared with the existing models, our new objective function distinguishes the clicked (query, document)-pairs with different relevance information, and our new parametrization of the user click probability considers both the semantic similarity and the term or lexical match information as the reasons for click(s). We tested our model on the media search logs of a commercial search engine and obtained superior performance under several metrics for relevance ranking.