Distribution-Aware Mutation Analysis

Mutation analysis is widely employed to evaluate the effectiveness of various software testing techniques. In most situations, mutation operators are uniformly applied to the original programs, while the faults tend to be clustered in practice. This may result in the inappropriate simulation of faults, and thus cannot deliver the reliable evaluation results. To overcome this, we propose a distribution-aware mutation analysis technique and conducted empirical studies to investigate the impact of the mutation distribution on the effectiveness evaluation of testing techniques. As an illustration, we select the commonly practiced random testing technique and two versions of dynamic random testing techniques and apply them to testing Web services. Results of empirical studies suggest that the mutation distribution significantly affects the evaluation results. This observation further indicates that the effectiveness of testing techniques previously evaluated with the uniform mutation analysis needs further realignments.