HCI⁁2 Workbench: A development tool for multimodal human-computer interaction systems

In this paper, we present a novel software tool designed and implemented to simplify the development process of Multimodal Human-Computer Interaction (MHCI) systems. This tool, which is called the HCI^2 Workbench, exploits a Publish / Subscribe (P/S) architecture to facilitate efficient and reliable inter-module data communication and runtime system management. In addition, through a combination of SDK, software tools, and standardized description / configuration file semantics, the HCI^2 Workbench provides an easy-to-follow procedure for developing highly flexible and reusable modules. Moreover, the HCI^2 Workbench features system persistence and portability by using standardized module packaging method and system configuration files. Last but not least, usability was another major concern. Unlike other similar tool, including Psyclone and ActiveMQ, the HCI^2 Workbench provides a complete graphical environment to support every step in a typical MHCI system development process, including module program development and debugging, module packaging, module management, system configuration, module and system testing, in a convenient and intuitive manner. To help demonstrating the HCI^2 Workbench, we also present a readily-applicable system developed using our tool. This open-source demo system, which is called the CamGame, consists of an interactive system allowing users to play a computer game using hand-held marker(s) and low-cost camera(s) instead of keyboard and mouse.