Multi-model approach based on 3D functional features for tool affordance learning in robotics

Tools can afford similar functionality if they share some common geometrical features. Moreover, the effect that can be achieved with a tool depends as much on the action performed as on the way in which it is grasped. In the current paper we present a two step model for learning and predicting tool affordances which specifically tackles these issues. In the first place, we introduce Oriented Multi-Scale Extended Gaussian Image (OMS-EGI), a set of 3D features devised to describe tools in interaction scenarios, able to encapsulate in a general and compact way the geometrical properties of a tool relative to the way in which it is grasped. Then, based on these features, we propose an approach to learn and predict tool affordances in which the robot first discovers the available tool-pose categories of a set of hand-held tools, and then learns a distinct affordance model for each of the discovered tool-pose categories. Results show that the combination of OMS-EGI 3D features and multi-model affordance learning approach is able to produce quite accurate predictions of the effect that an action performed with a tool grasped on a particular way will have, even for unseen tools or grasp configurations.