A sign reading system for the visually impaired

Navigating within new or unfamiliar environments is a major problem for the visually impaired, not only in moving about safely but also in finding their location and the way to their destination. One key issue is that the visually impaired cannot take advantage of available directional signs, maps, building directories, etc. Accordingly, we propose in this paper a helper system that aims at locating indoor signs and reading their content out loud. While existing approaches are limited by relying on specific shapes or color marks, or requiring user assistance, our system is able to automatically locate and read the signs present in the indoor environment. Using a portable camera, computer, and headphones, the system continuously captures images, locates and extracts the text within, and converts it into speech. The system integrates proven text processing and speech synthesis algorithms, as well as novel techniques for text identification and logical content filtering.