An Assessment of the Quality of Automated Program Operator Repair

Automated program repair (APR) techniques fix faults by repeatedly modifying suspicious code until a program passes a set of test cases. Although generating a repair is the goal of APR, a repair can have negative consequences. The quality of a repair is reduced when the repair introduces new faults and/or degrades maintainability by adding irrelevant but functionally benign code. We used two APR approaches to repair faulty binary operators: (1) find a repair in existing code by applying a genetic algorithm to replace suspicious code with other existing code as done by GenProg, and (2) mutate suspicious operators within a genetic algorithm. Mutating operators was clearly more effective in repairing faulty operators than using existing code for a repair. We also evaluated the approaches in terms of two potential negative effects: (1) the introduction of new faults and (2) a reduction of program maintainability. We found that repair processes that use tests that satisfy branch coverage reduce the number of new faults. In contrast, repair processes using tests that satisfy statement coverage and randomly generated tests introduce numerous new faults. We also demonstrate that a mutation based repair process produces repairs that should be more maintainable compared to those produced using existing code.