Predicting fault diagnosis performance: why are some bugs hard to find?

An experiment to examine how a troubleshooter´s mental model of a logic network might prejudice his or her ability to diagnose previously unseen faults is discussed. Subjects first inferred the structure of a logic network by viewing it in different states. Most subjects ended up with mental models of the network that were almost, but not completely, correct. Then, in a fault diagnosis phase, subjects diagnosed faults placed in the network by the experimenter. Faults involved the addition of new links or the deletion of existing links. In generating faults for subjects, two factors were varied in a 2×2 design: whether a subject believed a link existed or not and whether that belief was true or false. The ability to detect the symptoms of a fault was not always a guarantee of correct fault diagnosis. Subjects had difficulty diagnosing faults to do with links they believed did not exist, and they found it impossible to diagnose faults that influenced parts of the network about which they had false beliefs