Semantic-Based Test Oracles

Test oracle is one of the most difficult parts for test automation. For software with a large number of test cases, it is always both expensive and error prone to develop and maintain test oracles. The research is motivated by industry needs of automated testing on software with standard interfaces in an open system architecture. In counter to test oracle challenges, it proposes an innovative method to represent and calculate test oracles based on the semantic model of standard interface service specification of the software under test (SUT). Semantic model provides well-defined domain knowledge of service data, functionalities and constraints. Rules are created to model the expected SUT behavior in terms of antecedents and consequents. For each service, it captures both direct input-output relations and service interactions, that is, how the execution of a service may be affected by (pre-condition) or impact (post-condition) the SUT system state. As rule languages are neutral to programming languages, oracles specified in this way are independent of SUT implementations and can be reused across different systems conforming to the same interface standards. With the support of semantic techniques and tools like ontology modeler and rule engine, the proposed approach can enhance test oracle automation based on sophisticated defined domain model. Experiments and analysis show promising improvements in test productivity and quality.