Reaching and grasping novel objects: Using neural dynamics to integrate and organize scene and object perception with movement generation

We present a neural dynamics architecture for robotic grasping of novel objects. It closes the perception-action loop by integrating perceptual processes such as scene exploration, pose estimation, and shape classification with movement generation to reach and grasp a target object. Inspired by theories of human embodied cognition, this is achieved by interconnected dynamical systems, whose dynamical instabilities mark the discrete events of the grasping process. The architecture perceives the scene through a Kinect sensor and executes the grasp with a Schunk Dextrous Hand attached to a Kuka light weight arm.