Depth-Sensing Video Cameras for 3D Tangible Tabletop Interaction

Recently developed depth-sensing video camera technologies provide precise per-pixel range data in addition to color video. Such cameras will find application in robotics and vision-based human computer interaction scenarios such as games and gesture input systems. We present an interactive tabletop system which uses a depth-sensing camera to build a height map of the objects on the table surface. This height map is used in a driving simulation game that allows players to drive a virtual car over real objects placed on the table. Players can use folded bits of paper, for example, to lay out a course of ramps and other obstacles. A projector displays the position of the car on the surface, such that when the car is driven over a ramp, for example, it jumps appropriately. A second display shows a synthetic graphical view of the entire surface, or a traditional arcade view from behind the car. Micromotorcross is a fun initial investigation into the applicability of depth-sensing cameras to tabletop interfaces. We present details on its implementation, and speculate on how this technology will enable new tabletop interactions.