Robust direct visual servo using network-synchronized cameras

A direct visual servoing system is described which employs a network of cameras providing high-speed vision feedback that is robust to occlusions. This system does not rely on any external position or velocity sensors, but directly sets motor current using visual feedback alone. The limitation of 60 Hz video is overcome with multiple RS-170 cameras, synchronized over a network in round-robin fashion, to capture video fields at different instants in time. Each camera has its own computer that processes video at field rates to determine the position of a planar robot joint using eigenspace methods. The eigenspace computations produce position and Euclidean distance measurements sent from each camera node over a network to a master servo computer. It is shown that the Euclidean distance from the manifold in eigenspace in the presence of random occlusions is statistically related to the position measurement error. Occlusions are thus considered as "noise," and the measurement error variance is estimated directly from Euclidean distance. The measurement error variance is applied directly to a Kalman filter, which weights feedback from each camera to provide improved position estimates. The Kalman filter also models the vision transport delays to provide timely position estimates to ensure the stable direct visual servoing of a planar robot. Simulation results illustrate improvement in dynamic performance as the number of cameras are increased. Experimental measurements were obtained for a network of four cameras performing direct visual servoing of a simple planar robot. The results demonstrate the step response, as well as stable servo-hold operation in the presence of full occlusions in a subset of cameras or with partial occlusions in all cameras.