Sensor Fusion for Intuitive Robot Programming

Fusion of information from multiple sensors can greatly enhance the performance of human-machine interaction, especially in the intuitive robot programming. The methods aim to allow rapid teaching of robotic tasks in a safe and efficient manner. The techniques can reduce the setup time of a robotic system. This is crucial for SMEs (Small and Medium Enterprise) where the products in the manufacturing area are in small lot size but with high batch mix. The objective of this research is to fuse the information from a range sensor and a camera. An unique method using the surface constraint has been adopted for the calibration of the sensor fusion system. By taking the surface normal of a calibration board as the common feature, the transformation between the two coordinate systems can be formulated. The end result is a fused scene with both range and texture (color in this case) information. The range information will be used for the path generation for robotic tasks. On the other hand, the images captured by the camera together with the graphical user interface provide an user friendly interface platform for the user. As the two images have been fused, the operator can program a path for a robot to execute by ´point-and-click´ on the user interface screen. Experimental results have shown that the new method of robot programming, with sensor fusion information, has improved the robotic teaching process by at least 90% as compared to the manual programming method using teaching pendant.