Snap and Translate Using Windows Phone

Author

Du, Jun ; Huo, Qiang ; Sun, Lei ; Sun, Jian

Author_Institution

Microsoft Res. Asia, Beijing, China

fYear

2011

fDate

18-21 Sept. 2011

Firstpage

809

Lastpage

813

Abstract

We have developed a prototype of a mobile app called "Snap and Translate" on "Windows Phone 7". A person who is reading an English menu/sign and wants a Chinese translation of an English word or phrase or paragraph can use a Windows Phone to snap an image of the text, tap the word or swipe the phrase or circle the paragraph with a finger, and get a Chinese translation displayed on the screen of the phone. This is enabled by seamless integration of three Microsoft technologies: intelligent text extraction, OCR, and machine translation based on a client-plus-cloud architecture. The current prototype also supports Chinese OCR plus Chinese-to-English translation. In this paper, we highlight the UI design of the system and the corresponding user-intention guided text extraction approach to achieving a compelling user experience.

Keywords

cloud computing; document image processing; language translation; mobile computing; natural language processing; user interfaces; Chinese OCR; Chinese translation; Chinese-to-English translation; English menu; English sign; UI design; Windows Phone; client-plus-cloud architecture; intelligent text extraction; machine translation; mobile app; snap-and-translate; user-intention guided text extraction; Cameras; Dictionaries; Image color analysis; Image edge detection; Mobile communication; Optical character recognition software; Robustness; augmented reality; camera-based OCR; mobile translation; text detection; user interface;

fLanguage

English

Publisher

ieee

Conference_Titel

Document Analysis and Recognition (ICDAR), 2011 International Conference on

Conference_Location

Beijing

ISSN

1520-5363

Print_ISBN

978-1-4577-1350-7

Electronic_ISBN

1520-5363

Type

conf

DOI

10.1109/ICDAR.2011.166

Filename

6065423