BPMN Conformance in Open Source Engines

Service-oriented systems are increasingly implemented in a process-based fashion. Multiple languages for building process-based systems are available today, but the Business Process Model and Notation (BPMN) is becoming ubiquitous. With BPMN 2.0 released in 2011, execution semantics were introduced, supporting the definition of executable processes. Nowadays, more and more process engines directly support the execution of BPMN processes. However, the BPMN specification is lengthy and complex. As there are no official tests and no certification authority, it is very likely that engines a) implement only a subset of the language features and b) implement language features differently. In other words, we suspect that engines do not conform to the standard, despite the fact that they claim support for it. This prohibits the porting of processes between different BPMN vendors, which is an acclaimed goal of the language. In this paper, we investigate the standard conformance of open source BPMN engines to provide a clear picture of the current state of the implementation of BPMN. We develop a testing approach that allows us to build fully BPMN-compliant tests and automatically execute these tests on different engines. The results demonstrate that state of-the-art BPMN engines only support a subset of the language. Moreover, they indicate that porting BPMN processes is only feasible when using basic language constructs.