Online Segmentation and Classification of Manipulation Actions From the Observation of Kinetostatic Data

This paper presents an automated method for segmentation and classification of manipulation tasks. It introduces a method to build and update a dictionary of elementary actions, so as to express observed tasks as a sequence of items. Segmentation is carried out by splitting an observed manipulation task into submaneuvers. It is based on singular value decomposition of data that is gathered from the observation of humans. This observation consists of hand joint angles, the hand pose with respect to a world frame, and fingertip contact forces. The classification step introduces, from a large set of observed maneuvers, new entities called elementary actions that generalize the concept of segments, instances of elementary actions. This paper uses fingertip contact forces in the measured data. In grasping and manipulation tasks, the interaction between the hand and the object in the physical world is necessary to segment and interpret motion. A set of 120 maneuvers involving six tasks have been used to evaluate the methods with dependent measures including metrics of robustness, effectiveness, and repeatability. In such evaluations, the average value of the effectiveness metrics over all the maneuvers is 0.866. The interuser repeatability is equal to 0.8926, while the average repeatability is 0.911.