Handwritten and Audio Information Fusion for Mathematical Symbol Recognition

Considerable efforts are being done within the scientific community to make as easier as possible the way that the human being converses with its machine. Handwriting and speech are two common ways used to achieve this goal and are probably among those which attracted much interest. In mathematical content recognition tasks, these two modalities are used with a certain success. This paper presents an architecture based on a speech handwriting data fusion for isolated mathematical symbol recognition. Different fusion methods are explored. The results are very encouraging since recognition rates are increased comparatively to mono modality approaches.