Knowledge-based systems validation: when to stop running test cases

One of the unsettled problems in knowledge engineering and, particularly, in the field of validation is to determine when the validation process of a knowledge-based system is complete. Some hints are given in the literature, but little, if any, work has been done on how to address this problem from an analytical viewpoint. In this paper, the validation field is briefly surveyed, and we take a look at what the validation process should consist of. Then, an analytical model is proposed for obtaining the optimum number of test cases for validating the knowledge-based system under study. This mathematical model takes into account the v-type (knowledge-based system classed according to the validation process) defined in this paper, the expected degree of confidence in the validation process and previous results from similar knowledge-based system validation processes. It should be mentioned here that a theoretical result is given to solve eminently practical problem, a rare occurrence in the field of artificial intelligence as a whole and especially in knowledge engineering. © 1999 Academic Press