Balancing data-driven and rule-based approaches in the context of a multimodal conversational system

We address the issue of combining data-driven and grammar-based models for rapid prototyping of a multimodal conversational system. Moderate-sized rule-based spoken language models for recognition and understanding are easy to develop and provide the ability to prototype conversational applications rapidly. However, scalability of such systems is a bottleneck due to the heavy cost of authoring and maintenance of rule sets and inevitable brittleness due to lack of coverage in the rule sets. In contrast, data-driven approaches are robust and the procedure for model building is usually simple. However, the lack of data in an application context limits the ability to build data-driven models, especially in multimodal systems. We also present methods that reuse data from different domains and investigate the limits of such models in the context of an application domain.