A case study on regression test suite maintenance in system evolution

When a system is maintained, its automated test suites must also be maintained to keep the tests up to date. Even though practice indicates that test suite maintenance can be very costly we have seen few studies considering the actual efforts for maintenance of test-ware. We conducted a case study on an evolving system with three updated versions, changed with three different change strategies. Test suites for automated unit and functional tests were used for regression testing the extended applications. With one change strategy more changes were made in the tests code than in the system that was tested, and with another strategy no changes were needed for the unit tests to work.