Using Speech for Handwritten Mathematical Expression Recognition Disambiguation

Author

Medjkoune, Sofiane ; Mouchere, Harold ; Petitrenaud, Simon ; Viard-Gaudin, Christian

Author_Institution

IRCCyN, Univ. of Nantes, Nantes, France

fYear

2012

fDate

18-20 Sept. 2012

Firstpage

187

Lastpage

192

Abstract

The main goal of this work is to set up a multimodal system dedicated to mathematical expression recognition. In the proposed architecture, the transcription coming out from a speech recognition system is used to disambiguate the result of a handwriting recognition module. A set of keywords is built from the transcription module and used to rescore the outputs of both the handwriting classifier and the structural analysis module. Performances evaluated on the HAMEX dataset show a significant improvement over a single modality system.

Keywords

handwritten character recognition; image classification; speech recognition; text analysis; HAMEX dataset; handwriting classifier; handwriting recognition module; handwritten mathematical expression recognition disambiguation; keyword; multimodal system; single modality system; speech recognition system; structural analysis module; transcription module; Data integration; Databases; Educational institutions; Handwriting recognition; Speech; Speech recognition; Vocabulary; Data fusion; Handwriting recognition; Mathematical expression; Speech recognition;

fLanguage

English

Publisher

ieee

Conference_Titel

Frontiers in Handwriting Recognition (ICFHR), 2012 International Conference on

Conference_Location

Bari

Print_ISBN

978-1-4673-2262-1

Type

conf

DOI

10.1109/ICFHR.2012.288

Filename

6424390