Target object identification and localization in mobile manipulations

How to make mobile manipulator autonomously identify and locate target object in unknown environment, this is a very challenging question. In this paper, a multi-sensor fusion method based on camera and laser range finder (LRF) for mobile manipulations is proposed. Although the camera can acquire rich perceptual information, the image processing is very complex and easily influenced from the change in ambient light. Moreover, it can not directly provide the depth information of the environment. Since the LRF has the ability to directly measure 3D coordinates and the stability against the ambient light influence, meanwhile, the camera has the ability to acquire color information, the combination of the two sensors by making use of their advantages is utilized to obtain more accurate measurements as well as to simplify information processing. To overlay the camera image with the measurement points of the LRF pitching scan and to reconstruct the 3D image which includes the depth-of-field information, the model and the calibration of the system are built. Based on the combination of the color features extracted from the color image and the shape, size features extracted from the 3D depth-of-field image, the target object identification and localization are implemented autonomously. In order to extract the shape and size features, a triangular facet normal vector clustering (TFNVC) algorithm is introduced. The effectiveness of the proposed method and algorithm are validated by some experimental testing and analysis carried out on the mobile manipulator platform.