User-Oriented Evaluation Methods for Interactive Web Search Interfaces

Although significant efforts have been devoted to the study and evaluation of information retrieval systems from an algorithmic perspective, far less work has been performed on the evaluation of these systems from the user´s perspective. This is certainly the case for Web information retrieval, where the major search engines continue to utilise interfaces that have not changed substantially since their introduction. One of the challenges in developing new Web search interfaces is the evaluation of these systems in comparison to one another, as well as in comparison to the popular Web search engines. In this paper, we highlight some of the methods used in the literature for evaluating Web search systems, and present a summary of the methods that we have found to be effective in dealing with the challenges of evaluating intelligent and interactive Web search interfaces.