Integrated system interoperability testing with applications to VoIP

This work has been motivated by the need to test interoperability of systems carrying voice calls over the IP network. The voice over IP (VoIP) systems must be integrated and interoperate with the existing public switched telephone network (PSTN) before they are widely adopted. Standards have been developed to address this problem, but unfortunately different standards bodies and commercial consortiums have defined different standards. Furthermore, the prevailing VoIP standard such as H.323 is incomplete, complex, and presents the implementers with "vendors latitudes". As a result, there is no guarantee that the integrated VoIP systems would interoperate properly even if the implementations are all H.323-compliant. Thus interoperability testing has become indispensable. We want to test all the system interoperations by exercising all the required patterns of "interoperating behaviors". On the other hand, test execution in real environment is expensive, and we want to minimize the number of tests while maintaining the coverage. We present a general method for automatic generation of test cases, which cover all the required system interoperations and contain a minimal number of tests. We study data structures and efficient test generation algorithms, which take time proportional to the total test case size. Finally, we report experimental results on VoIP systems.