Extending Manual GUI Testing Beyond Defects by Building Mental Models of Software Behavior

Manual GUI testing involves providing inputs to the software via its GUI and determining the software´s correctness using its outputs, one of them being the GUI itself. Because of its human-in-the-loop nature, GUI testing is known to be a time-consuming activity. In practice, it is done by junior, inexpensive testers to keep costs low at the very tail-end of the software development process. In this paper, we posit that the importance of GUI testing has suffered due to its traditional narrow role -- to detect residual software defects. Because of its human-in-the-loop nature, GUI testing has the potential to provide outputs other than defects and to be used as inputs to several downstream activities, e.g., security analysis. One such output is the mental model that the GUI tester creates during testing, a model that implicitly informs the tester of the software designer´s intent. To evaluate our claim, we consider an important question used for security assessment of Android apps: "What permission-sensitive behaviors does this app exhibit?" Our assessment is based on the comparison of 2 mental models of 12 Android apps -- one derived from the app´s usage and the other from its public description. We compare these two models with a third, automatically derived model -- the permissions the app seeks from the Android OS. Our results show that the usage-based model provides unique insights into app behavior. This model may be an important outcome of GUI testing, and its consistency with other behavioral information about the app could later be used in software quality assurance activities such as security assessment.