The mobile manipulation system of the Household Butler Robot based on multi-monocular cameras

The Household Butler Robot (HBR) project is a long-term group effort aiming to develop a multifunctional and robust household assistant robot. Thus an efficient mobile manipulation system is the key component for this robot to be practically useful. In this paper a mobile manipulation system based on multi-view monocular cameras applied in our robot is described in detail. This systems mainly consists of a head camera, a pair of arm cameras and a pair of bimanual arms. By combining visual information acquired by these cameras, the whole grasping routine is split into four steps in a “Look-Move-Look-Move” pattern and be solved step by step. Also a series of experiments about grasping known objects within household environments are performed to validate and evaluate this system. These results illustrate that this system successfully and efficiently accomplishes grasping tasks.