Title :
Corroborating User Assessments of Software Behavior to Facilitate Operational Testing
Author :
Augustine, Vinay ; Podgurski, Andy
Author_Institution :
Case Western Reserve Univ., Cleveland
Abstract :
Operational or "beta" testing of software has a number of benefits for software vendors and has become common industry practice. However, ordinary users are more likely to overlook or misreport software problems than experienced software testers are. To compensate for this shortcoming, we present a technique called corroboration-based filtering for corroborating user assessments of individual operational executions for which audit information has been captured for possible offline review. Independent assessments concerning similar executions are pooled by automatically clustering together executions with similar execution profiles. Executions are chosen for review based on their user assessments, the size of the cluster each execution belongs to, and whether the cluster has already been confirmed by developers to contain an actual failure. We explain the rationale for this technique, analyze it probabilistically, and present the results of empirically comparing it to alternative techniques.
Keywords :
program testing; audit information; corroboration-based filtering; operational testing; similar execution profiles; software behavior; software problems; software testing; user assessments; Automatic testing; Computer industry; Computer science; Costs; Information filtering; Information filters; Internet; Reliability engineering; Software reliability; Software testing;
Conference_Titel :
Software Reliability, 2007. ISSRE '07. The 18th IEEE International Symposium on
Conference_Location :
Trollhattan
Print_ISBN :
978-0-7695-3024-6
DOI :
10.1109/ISSRE.2007.30