Geometric image masking for segmenting dynamic projected imagery

Projected interfaces are gaining popularity as a means of access to information anywhere in a space without the need for special devices or monitors. In this context, it is of particular interest to automatically steer the projected interface to appropriate surfaces based on the location of the user in the space. This requires the users be tracked precisely, and in real time, as they move about in the environment, in spite of continuously varying and moving projected imagery in the background. This is challenging as the projection regions also appear as foreground. This paper presents a geometric image masking technique that separately segments out regions corresponding to the user and projected imagery in multiple camera views, and enables precise three-dimensional tracking of the user.