Specification-based test oracles for reactive systems

The testing process is typically systematic in test data se- lection and test execution. For the most part, however, the effective use of test oracles has been neglected, even though they are a critical component of the testing process. Test or- acles prescribe acceptable behavior for test execution. In the absence of judging test results with formal oracles, testing does not achieve its goal of revealing failures or assuring cor- rect behavior in a practical manner; manual result checking is neither reliable nor cost-effective. We argue that test oracles should be derived from specifications in conjunction with testing criteria, represented in a common form, and their use made integral to the test- ing process. For complex, reactive systems, oracles must reflect the multiparadigm nature of the required behavior. Such systems are often specified using multiple languages, each selected for its utility specifying in a particular computational paradigm. Thus, we are developing an approach for deriving and using oracles based on multiparadigm and multilingual specifications to enable the verification of test results for reactive systems as well as less complex systems.