Development of object manipulation through self-exploratory visuomotor experience

Human infants learn to interpret their visuomotor experience and predict the effects of their actions by practicing interactions with the environment. This paper presents a computational model of this process that can be implemented in an artificial agent. We first present a mechanism for simultaneous segmentation and modeling of the agent´s body, movable objects and their visual environment. This model can explain a “sense of agency” in terms of predictive certainty of object movements conditioned by actions. We then describe causality learning for object manipulation in detail. Our experimental setup requires a model considering combinational causalities beyond simple direct causality. A novel strategy for causal exploration is proposed and its effectiveness is shown in experiments. The results show that the proposed model allows an agent to efficiently acquire object manipulation skills through self-exploratory visuomotor experience, that is, a sequence of pairs of raw bitmap image and taken actions at each time step.