DocumentCode :
62050
Title :
On the Accuracy, Efficiency, and Reusability of Automated Test Oracles for Android Devices
Author :
Ying-Dar Lin ; Rojas, Jose F. ; Chu, Edward T-H ; Yuan-Cheng Lai
Author_Institution :
Dept. of Comput. Sci., Nat. Chiao Tung Univ., Hsinchu, Taiwan
Volume :
40
Issue :
10
fYear :
2014
fDate :
Oct. 1 2014
Firstpage :
957
Lastpage :
970
Abstract :
Automated GUI testing consists of simulating user events and validating the changes in the GUI in order to determine if an Android application meets specifications. Traditional record-replay testing tools mainly focus on facilitating the test case writing process but not the replay and verification process. The accuracy of testing tools degrades significantly when the device under test (DUT) is under heavy load. In order to improve the accuracy, our previous work, SPAG, uses event batching and smart wait function to eliminate the uncertainty of the replay process and adopts GUI layout information to verify the testing results. SPAG maintains an accuracy of up to 99.5 percent and outperforms existing methods. In this work, we propose smart phone automated GUI testing tool with camera (SPAG-C), an extension of SPAG, to test an Android hardware device. Our goal is to further reduce the time required to record test cases and increase reusability of the test oracle without compromising test accuracy. In the record stage, SPAG captures screenshots from device´s frame buffer and writes verification commands into the test case. Unlike SPAG, SPAG-C captures the screenshots from an external camera instead of frame buffer. In the replay stage, SPAG-C automatically performs image comparison while SPAG simply performs a string comparison to verify the test results. In order to make SPAG-C reusable for different devices and to allow bettersynchronization at the time of capturing images, we develop a new architecture that uses an external camera and Web services to decouple the test oracle. Our experiments show that recording a test case using SPAG-C´s automatic verification is as fast as SPAG´s but more accurate. Moreover, SPAG-C is 50 to 75 percent faster than SPAG in achieving the same test accuracy. With reusability, SPAG-C reduces the testing time from days to hours for heterogeneous devices.
Keywords :
Web services; graphical user interfaces; program testing; smart phones; software reusability; Android application; Android devices; Android hardware device; DUT; GUI layout information; SPAG-C; Web services; automated test oracles; automatic verification; capturing images; device under test; event batching; frame buffer; image comparison; replay process; replay stage; smart phone automated GUI testing tool; smart wait function; test accuracy; test case writing process; traditional record-replay testing tools; user events; verification process; with camera; Accuracy; Androids; Graphical user interfaces; Humanoid robots; Performance evaluation; Smart phones; Testing; Reusable software; test execution; testing tools; user interfaces;
fLanguage :
English
Journal_Title :
Software Engineering, IEEE Transactions on
Publisher :
ieee
ISSN :
0098-5589
Type :
jour
DOI :
10.1109/TSE.2014.2331982
Filename :
6840332
Link To Document :
بازگشت