Title :
Snap and Translate Using Windows Phone
Author :
Du, Jun ; Huo, Qiang ; Sun, Lei ; Sun, Jian
Author_Institution :
Microsoft Res. Asia, Beijing, China
Abstract :
We have developed a prototype of a mobile app called "Snap and Translate" on "Windows Phone 7". A person who is reading an English menu/sign and wants a Chinese translation of an English word or phrase or paragraph can use a Windows Phone to snap an image of the text, tap the word or swipe the phrase or circle the paragraph with a finger, and get a Chinese translation displayed on the screen of the phone. This is enabled by seamless integration of three Microsoft technologies: intelligent text extraction, OCR, and machine translation based on a client-plus-cloud architecture. The current prototype also supports Chinese OCR plus Chinese-to-English translation. In this paper, we highlight the UI design of the system and the corresponding user-intention guided text extraction approach to achieving a compelling user experience.
Keywords :
cloud computing; document image processing; language translation; mobile computing; natural language processing; user interfaces; Chinese OCR; Chinese translation; Chinese-to-English translation; English menu; English sign; UI design; Windows Phone; client-plus-cloud architecture; intelligent text extraction; machine translation; mobile app; snap-and-translate; user-intention guided text extraction; Cameras; Dictionaries; Image color analysis; Image edge detection; Mobile communication; Optical character recognition software; Robustness; augmented reality; camera-based OCR; mobile translation; text detection; user interface;
Conference_Titel :
Document Analysis and Recognition (ICDAR), 2011 International Conference on
Conference_Location :
Beijing
Print_ISBN :
978-1-4577-1350-7
Electronic_ISBN :
1520-5363
DOI :
10.1109/ICDAR.2011.166
