An image processing method for use in a GUI for the visually impaired

This paper presents a method for processing the image of a human hand, in order to emulate the acts of moving a scroll bar by rotating the hand, and clicking on a mouse button by moving the thumb. This method can be used in a system that facilitates the operation of a graphical user interface (GUI) by a visually impaired person. The grayscale image acquired by a digitizer is thresholded and converted to a black-and-white image. The orientation of the hand, given by an angle θ with the vertical axis, is calculated based on the central moments. The image of the hand is then rotated by θ to a normalized position. The number of pixels in each column of the normalized image is counted, and the result is expressed in a histogram. The coefficient of asymmetry of this histogram is used to determine whether the thumb is positioned along the pointing finger or whether it is far from the other fingers, defining two states that correspond to a mouse button up or down. Experimental results showed a frame rate of 12.5 frames per second, demonstrating that this method is suitable for the real time processing demanded by the interface system