Semantic labelling to aid navigation in prosthetic vision

Current and near-term implantable prosthetic vision systems offer the potential to restore some visual function, but suffer from limited resolution and dynamic range of induced visual percepts. This can make navigating complex environments difficult for users. Using semantic labelling techniques, we demonstrate that a computer system can aid in obstacle avoidance, and localizing distant objects. Our system automatically classifies each pixel in a natural image into a semantic class, then produces an image from the induced visual percepts that highlights certain classes. This technique allows the user to clearly perceive the location of different types of objects in their field of view, and can be adapted for a range of navigation tasks.