User-Oriented Evaluation of Event-Based Decision Support Systems

Artificial Intelligence-based event recognition systems carry high potential for organisations to utilise their structured and unstructured data. The application of these systems as a backbone of decision support systems allows for effective and efficient information management. To sufficiently evaluate such kind of integrated systems recognising events for the benefit of decision makers, a holistic methodology is necessary. We propose a new methodology which complements existing approaches for technology-oriented verification and validation by user-oriented evaluation (user experience analysis). We illustrate the proposed methodology by evaluating EP-IRM, an event processing system for intelligent resource management. This case study shows that our methodology offers invaluable information about the performance and acceptance of an event based decision support system.