A multimodal, keyword-based spoken dialogue system-MultiksDial

In this paper, a multimodal, keyword-based, spoken dialog system (“MultiksDial”) is described. The system provides multiple input channels of spontaneous speech and designation by touch, as well as multiple output channels of graphics and voice responses by text-to-speech. The system also provides three types of sensors to detect the user´s actions and to plan interactive strategies. A word spotter handles real-time word spotting on medium-sized vocabulary words. The multimodal interaction mechanism is evaluated on a directory guidance system. The experimental results showed comparative merits of usability against an ordinary touch-screen system. Multimodal input is especially effective for novice users. Cooperative guidance using sensors, graphics, and speech also helped novice users. A multimodal UI development tool has been developed for rapid prototyping of MultiksDial