Issues in Testing Dependable Event-Based Systems at a Systems Integration Company

Testing of dependable event-based systems is very important to ensure that all requirements (including nonfunctional requirements such as reliability, availability, safety and security) are met and the relevant standards are considered. Siemens Program and Systems Engineering is a company that builds dependable event-based systems in multiple domains. A special unit at PSE, the Support Center Test, focuses on testing issues. In this paper we provide an overview of the state-of-the-practice in testing dependable event-based systems and identify the challenges that have to be addressed in the future. We illustrate our findings by two case studies, one for a transportation system and one for telecommunications. The top three topics for improvement are: testability and requirements tracing, generation of test cases, and verification and validation