Development of an information projection interface using a projector-camera system

This paper proposes a position-free interface system that enables a user to input and obtain information everywhere in the environment with a projector-camera system. Hand-waving detected in the image is used as a cue to start the system, to distinguish gestures based on the user´s intension of operation from other daily-life motions, and to specify the position of the user´s hand. Hand-waving detection is implemented based on combination of background subtraction method and skin color extraction. In addition, we focus on changes of brightness value in each pixel. The user´s hand is tracked using a particle filter algorithm. The user designates the projection area freely on a plane in the environment with his/her hand. The projector mounted on a pan-tilt unit projects information according to the user´s designation. Then, the user´s fingertip is detected using simple template matching and he/she interacts with the projected information to input commands without any devices. As one of applications, we have developed an interface system to operate electrical appliances (TV) intuitively in a daily life scene.