Developing and testing algorithms for stopping testing, screening, run-in of large systems or programs

When large hardware-software systems are run-in or an acceptance testing is made, a problem is when to stop the test and deliver/accept the system. The same problem exists when a large software program is tested with simulated operations data. Based on two theses from the Technical University of Denmark, the paper describes and evaluates 7 possible algorithms. Of these algorithms, the three most promising are tested with simulated data. 27 different systems are simulated, and 50 Monte Carlo simulations made on each system. The stop times generated by the algorithm is compared with the known perfect stop time. Of the three algorithms two is selected as good. These two algorithms are then tested on 10 sets of real data. The algorithms are tested with three different levels of confidence. The number of correct and wrong stop decisions are counted. The conclusion is that the Weibull algorithm with 90% confidence level takes the right decision in every one of the 10 cases