Evaluating the Effect of Test Case Length on GUI Test Suite Performance

In this paper we investigate the effect of test case length on the performance of test suites for automated graphical user interface (GUI) testing. Research has been done on the effect of test length for traditional white-box testing, however, these findings do not necessarily extend to GUI testing. We run experiments on five subject applications using a custom-built dynamic test generator. We measure the effect that test length has on statement, branch, and event-pair coverage, as well as the fault-finding abilities for each test suite. Our findings indicate that longer test suites perform better than shorter test suites in terms of coverage, fault-finding, and cost. However, the effect eventually diminishes with greater increase in length. As well, the results vary depending on the application under test.