Risk-Based Stopping Criteria for Test Sequencing

Testing complex manufacturing systems, like ASML lithographic machines, can take up to 45% of the total development time. The decision of when to stop testing is often difficult to make because less testing may leave critical faults in the system, while more testing increases time-to-market. In this paper, we solve the problem of deciding when to stop testing by introducing a test-sequencing method that incorporates several stopping criteria. These stopping criteria consist of objectives and constraints on the test cost and the remaining risk cost. For a given problem, a suitable stopping criterion can be chosen. For example, with the risk-based stopping criterion, testing stops when the test time or cost exceeds the risk cost. Furthermore, we show that it also is possible to model reliability problems with this test-sequencing method. The method is demonstrated on ASML systems with two case studies. The first case study was conducted in the test phase during the development of the software that is used to control an ASML lithographic machine. The second case study was conducted on the reliability testing of a lithographic machine.