Discovering text databases on the Internet: neural net agent approach

The proliferation of readily available but rapidly increasing document databases on the Internet has led to the text database discovery problem: finding those text databases (out of many candidates) that provide documents that are relevant to the user query. In this paper, we present a neural net agent for text database discovery. Our agent, which is trained with sufficient training queries, discovers those text databases that contain the relevant documents for a given query and then retrieves those documents effectively. We first present a framework for the neural net agent, and then describe its training and document retrieval procedures. We also evaluate the performance of our neural net agent experimentally