3D Position based multiple human servoing by low-level-control of 6 DOF industrial robot

In this paper, we present a new vision-based multiple human tracking system. This novel 3D visual tracking system is capable of automatically identifying, labeling and tracking multiple humans in real-time even when they occlude each other. Furthermore, the multiple human tracker was implemented in a vision driven robot system for human robot interaction. The distributed system comprises of 4 subsystems: a)Multiple Human Tracking System, b) Robot Control System, c) 3D Visualization System and d) Remote Interface System. The Visual Tracking System performs real-time detection and tracking of humans in 3D within a large workspace. The Robot System uses the 3D position data of the targets obtained from the vision system to interact with the humans. The visual information is also used to monitor safe interaction within humans and robot. The Robot System is a 6DOF Stäubli TX90 industrial arm, controlled in real-time through a low-level interface. A real-time representation of the actual environment is rendered in 3D by the 3D Visualization System. The individual subsystems communicate with each other over a common communication engine based on TCP/IP. The complete system can be controlled and monitored through a wireless device.