Advanced decision-making systems in future avionics: automatic target recognition example

In future avionics systems, there will be a stronger and stronger pull for more automation of decision-making functions. The nature of these decision-making functions will be of such complexity that exhaustive testing of all possible decisions will be impossible. Further, the complexity will also preclude exhaustive “training” of the decision-making systems such as the training required of learning-based paradigms like neural networks. The primary thesis of this paper advocates a change of focus in training and testing procedures for advanced decision-making systems such as automatic target recognizers (ATRs). Whereas currently, the predominant state of the art in ATR testing is preoccupied on making sure that the training and testing data come from the same population. This common procedure helps promote excellent results but does not provide insight into performance of the system under real world conditions. Rather, the focus should be on whether the testing is representative of the real world or whether it even scales to the complexity of the real world situations. Testing performed in this way will result in different approaches to decision-making systems such as ATR and, it is believed, will accelerate progress in ATR and other decision-making functions