Assessment of the reliability of AI programs

Analytical tools for assessing the reliability of AI (artificial intelligence) programs are developed. It is shown that conventional software reliability models must be modified to incorporate certain special characteristics of AI programs, such as: failures due to intrinsic faults, e.g., limitations due to heuristics and other basic AI techniques; a fuzzy correctness criterion, i.e., the difficulty in accurately classifying the output of some AI programs as correct or incorrect; computation time versus response time tradeoffs; and reliability growth due to an evolving knowledge base. The authors illustrate the approach by modifying the Musa-Okumoto (J.D. Musa and K. Okumoto, 1984), software reliability growth model to incorporate failures due to intrinsic faults and to accept fuzzy failure data. They also analyze in detail the reliability of heuristics programs in real-time situations and show that under certain conditions the cost based A* algorithm is less reliable than the node-based A* algorithm