Development of an evaluation model for Question Answering Systems

Question Answering Systems (QAS) are receiving increasing attention from information systems researchers, particularly those in the information retrieval and natural language processing communities. Evaluation of an information system’s success and user satisfaction are important issues in information systems research, especially for emerging online service systems on the Internet. Although many QAS have been implemented, little work has been done on the development of an evaluation model for them. The purpose of this study is to develop an evaluation model for question answering systems from user’s perspective. We have established the theoretical foundation and conceptualization of the constructs for user satisfaction with question answering systems. We have developed a general instrument capable of reliably and accurately measuring user satisfaction in the context of question answering systems. The proposed evaluation model provides a framework for the design of question answering systems from the user’s perspective to enhance user satisfaction and acceptance of QAS.