Tracking and Predictive Display for a Remote Operated Robot using Uncalibrated Video

Delays in the visual feedback can seriously impact operator performance in telerobotics. In predictive display synthesized visual feedback is rendered immediately in response to operator motions. In this paper we present a system using a geometric and appearance model that is captured using structure-from-motion by an uncalibrated camera. The geometric model is integrated into a registration based tracking algorithm that allows stable tracking of full 3D pose of the robot. Experimentally we show that predictive scene views can be rendered with both high visual fidelity and metric accuracy.