A competition-based connectionist model for information retrieval

In this paper, we adapt a competition-based connectionist model, which has been proposed for diagnostic problem solving, to information retrieval. In our model, documents are treated as “disorders” and user information needs as “manifestations”, and a competitive activation mechanism is used which converges to a set of disorders that best explain the given manifestations. By combining the ideas of Bayesian inferencing and diagnostic inferencing using parsimonious covering theory, this model removes many difficulties of direct application of Bayesian inference to information retrieval, such as the unrealistically large number of conditional probabilities required as part of the knowledge base, the computational complexity, and unreasonable independence assumptions. Also, Bayesian inference strengthens the parsimonious covering model by providing a likelihood measure which can be used to rank documents as well as to guide the search to the most likely retrieval. We also incorporate two types of relevance information to improve the model. First, Roget´s thesaurus is used to provide semantic relevance information among the index terms. Second, after the neural network has been initialized, it is trained using the available query-document relevance judgements. Our preliminary study demonstrate the efficiency and the retrieval precision of this model, comparable to or better than that of the Bayesian network models reported in the literature