Robust manipulation for temporary lack of sensory information by a multi-fingered hand-arm system

This paper proposes a novel vision-based grasping and manipulation scheme of a multi-fingered hand-arm system robust for a temporary lack of sensory information. Visual information is one of the fundamental components for reliable grasping and manipulation by a multi-fingered hand-arm system. However, in case that visual information such as position and attitude of an object comes to be unavailable due to the occlusion or if the object goes out-of-sight temporarily, unstable and unfavorable behavior is often induced. The proposed method, which utilizes the stable grasping control and the concept of virtual frame, enables to grasp and manipulate an object stably even if the visual information becomes suddenly and temporarily unavailable during manipulation. Firstly, a dynamical model of object grasping using a multi-fingered hand-arm system is formulated. Next, a new control scheme for robust object grasping and manipulation using the virtual frame is proposed. Finally, numerical simulations are performed to verify the usefulness of the proposed method.