Constructing transition models of AI planner behavior

Evaluation and debugging of AI systems require coherent views of program performance and behavior. We have developed a family of methods, called Dependency Detection, for analyzing execution traces for small patterns. Unfortunately, these methods provide only a local view of program behavior. The approach described here integrates two methods, dependency detection and CHAID-based analysis, to produce an abstract model of system behavior: a transition diagram of merged states. We present the algorithm and demonstrate it on synthetic examples and data from two AI planning and control systems. The models produced by the algorithm summarize sequences and cycles evident in the synthesized models and highlight some key aspects of behavior in the two systems. We conclude by identifying some of the inadequacies of the current algorithm and suggesting enhancements.