Object concept modeling based on the relationship among appearance, usage and functions

In this paper, a novel object concept model, which encodes the relationship among appearance, functions and usage, is proposed. The essential attribute of an object (artifact) is its function that achieves a particular purpose. Therefore, the function model is constructed through observations from a camera at first. The function is defined as changes in the work object before and after tool use. At the same time, the usage model is constructed from observations of the hand shape, grasping parts, and contact points of the tool. And then, the proposed system learns the object concept that is based on the relationship among appearance, and learnt function and usage models. The object appearance is represented by SIFT (Scale Invariant Feature Transform). Since the proposed models are based on the graphical model, it is possible for the system to stochastically infer unobservable information from observed one. For example, the system can infer usage and/or functions of the tool visually through the proposed model. Some experimental results using the system, in which the proposed model is implemented, are shown to validate the proposed model.