Interacting with Projected Media on Deformable Surfaces

This paper presents a novel human-computer interface for projector/camera-based applications that uses a deformable interaction surface. We discuss its design and implementation within the context of a radically different approach for controlling home appliances by pressing virtual buttons that are projected on soft deformable surfaces such as a sofa pillow. Effective real-time computer vision algorithms for implementing pointing and selection action detection for such an interface are discussed. Experimental results highlight the parameters and factors that have significant effect on the overall performance of such an interface.