Evaluating workflow management systems

We describe a project focusing on the evaluation of workflow management systems for a large Austrian bank. In the first part we establish a catalogue of evaluation criteria. It summarizes the huge amount of criteria into smaller classes (criteria categories) and orders the classes in a semantically structured directory. In the second part, we describe a tool implementing the evaluation framework. A database stores the criteria classes along with associated questions as well as the appropriate answers for a set of workflow products. The tool allows to extend the database as well as to perform a customized evaluation process. In the third part of the paper, we describe some of the results of using the tool for evaluating workflow management systems for the First Austrian Savings Bank.