Direct visual servoing using network-synchronized cameras and Kalman filter

A distributed camera system using off-the-shelf components is presented that demonstrates the capability to perform high-speed vision feedback suitable for applications such as direct visual servoing. The limitation of 60 Hz video sample rates is overcome by using multiple RS-170 cameras synchronized over a network to capture at different instants in time. Each camera node has its own computer that processes video at field rates to determine the pose of a planar robot joint using eigenspace methods. Position information is fed back over a network to a master computer to perform direct visual servoing. The resulting vision feedback from the multiple cameras uses a Kalman filter to estimate position and to model the vision computation and transport delays. Computer simulation results are provided as the number of cameras are varied. Finally, real-time experimental results are presented that verify the approach using a network of four cameras performing direct visual servoing of a simple planar robot.