A robust mobile robot manipulation system using a switch-based visual-servo controller

This paper develops a robust vision-based mobile manipulation system for wheeled mobile robots. In particular, the paper addresses the retention of visual features in the field of view of the camera, which is an important robustness issue in visual servoing. First, the classical approach of Image-Based Visual Servoing (IBVS) for fixed-base manipulators is extended to wheeled mobile robots. Second, in order to guarantee visibility of visual features, an innovative controller with machine learning using Q-learning is proposed, which can learn its behavior policy and autonomously improve its performance. Third, a hybrid controller for robust mobile manipulation is developed to integrate the IBVS controller and the Q-learning controller through a rule-based arbitrator. This is thought to be the first work that integrates Reinforcement Learning or Q-learning with visual servoing to achieve robust operation. Experiments are carried out to validate the approaches developed in this paper. The experimental results show that the new hybrid controller developed here possesses the capabilities of self-learning and fast response, and provides a balanced performance with respect to robustness and accuracy.