Multimodal integration in the MUeSLI project

The rapid acceptance of the World Wide Web (WWW) has led to a lot of interest in the use of online shopping or e-commerce. This is driving forward new interface designs which offer a more natural and human-like interaction. MUeSLI is a system which allows a user to browse through fabrics and apply them to a virtual sitting room by talking to a synthetic persona and touching objects and fabrics in the room. The paper outlines the programme and progress of the MUeSLI project currently being undertaken at BT Laboratories. The main focus of the project is the integration of two or more modes of interaction within a real-world application. The application has been through a number of iterations: a focus group, Wizard of Oz trials and is currently being developed into a fully automatic system. The development of the architecture has highlighted the complexities surrounding the interpretation of more than one mode, and a number of pertinent issues are discussed. Further trials, eventually using members of the public, will allow us to investigate the effective design of robust interfaces