Evaluation of Maintenance Software in Real-Time Systems

Statistical theory can be applied to data collected by physical fault insertion of small, random fault samples to estimate the critical parameters of software responsible for fault recovery, detection, and resolution in real-time systems. Estimates of the critical parameters can be used to determine if system reliability objectives are satisfied. This method of software evaluation, modified because of the impracticality of selecting and physically inserting truly random fault samples, was verified against a diagnostic/ Trouble Locating Program (TLP) whose critical parameters were known and was then used to evaluate and optimize a new diagnostic/ TLP program of unknown quality. Empirical evidence indicates that there is a close correlation between system performance against real faults and against the selected subset of real faults used for sampling.