Diversity oriented test data generation using metaheuristic search techniques

We present a new test data generation technique which uses the concept of diversity of test sets as a basis for the diversity oriented test data generation – DOTG. Using DOTG we translate into an automatic test data generation technique the intuitive belief that increasing the variety, or diversity, of the test data used to test a program can lead to an improvement on the completeness, or quality, of the testing performed. We define the input domain perspective for diversity (DOTG-ID), which considers the distances among the test data in the program input domain to compute a diversity value for test sets. We describe metaheuristics which can be used to automate the generation of test sets for the DOTG-ID testing technique: simulated annealing; a genetic algorithm; and a proposed metaheuristic named simulated repulsion. The effectiveness of DOTG-ID was evaluated by using a Monte Carlo simulation, and also by applying the technique to test simple programs and measuring the data-flow coverage and mutation scores achieved. The standard random testing technique was used as a baseline for these evaluations. Results provide an understanding of the potential gains in terms of testing effectiveness of DOTG-ID over random testing and also reveal testing factors which can make DOTG-ID less effective.