Statistical Sampling Based Approach to Alleviate Log Replay Testing

Typical software systems in financial firms are message- based server-side systems , the regression testing of which is currently done through a ´log- replay´ mechanism. As the production volumes are large and continue to grow rapidly, this approach leads to many problems such as scalability, resource crunch, reconciliation, redundancy, production impact. The approach in this paper continues to use the production log as the basis for regression testing. However, it deviates from the current approach in not using the entire production log but only a subset or sample of it.This approach has been implemented for regression testing of production systems in a major financial firm. The result was that the number of test cases executed converged to a small fraction of the number previously being executed, while still retaining a high degree of confidence in the code- coverage quality of the selected sample.