Title :
Combining stochastic and grammar-based language processing with finite-state edit machines
Author :
Johnston, Michael ; Bangalore, Srinivas
Author_Institution :
AT&T Labs-Res., NJ
Abstract :
Multimodal grammars provide an expressive formalism for rapid bootstrapping of finite-state mechanisms for multi-modal integration and understanding. These mechanisms align speech and gesture inputs, readily scale to processing of lattice inputs, and enable recovery from speech and gesture recognition errors through mutual compensation. However, in common with other handcrafted mechanisms, they can be brittle with respect to unexpected, erroneous, or disfluent inputs. In this paper, we show how the robustness of stochastic language models can be combined with the expressiveness of multimodal grammars by adding a finite-state edit machine to the multimodal language processing cascade. We evaluate the effectiveness of the approach in a multimodal conversational system (MATCH) which provides restaurant and subway information on a speech and pen enabled mobile device
Keywords :
finite state machines; gesture recognition; grammars; natural languages; speech recognition; finite-state edit machines; gesture recognition errors; grammar-based language processing; multimodal conversational system; multimodal grammars; speech recognition; stochastic language models; Cities and towns; Displays; Lattices; Natural languages; Robustness; Speech analysis; Speech processing; Speech recognition; Stochastic processes; Transducers;
Conference_Titel :
Automatic Speech Recognition and Understanding, 2005 IEEE Workshop on
Conference_Location :
San Juan
Print_ISBN :
0-7803-9478-X
Electronic_ISBN :
0-7803-9479-8
DOI :
10.1109/ASRU.2005.1566479