Mutation testing in practice using Ruby

Mutation testing is an effective testing technique to detect faults and improve code quality. However, few practitioners have adopted mutation testing into practice, which raises several questions: Are tests capable of killing mutants useful? What is the main hindrance to adopting mutation testing in practice? Can practitioners really integrate mutation testing into real-world agile development processes? In this paper, we present two major contributions. First, based on our analysis and knowledge of Ruby, we devised eight new mutation operators for Ruby. Second, we applied mutation testing to an industrial Ruby project at Medidata and reported the lessons learned from the study. We confirmed that mutation-adequate tests are useful and could improve code quality from the perspective of practitioners and found long mutation execution time hinders the agile process. In addition, we used an enterprise-level Amazon cloud-computing technique to reduce the computational cost of running mutants. Considering the availability of a mutation testing tool with our suggested features, we argue that mutation testing can be used in practice.