Learning interactions among objects, tools and machines for planning

We propose a method for learning interactions among objects when intermediate state information is not available. Learning is accomplished by observing a given sequence of actions on different objects. We have selected the Incredible Machine game as a suitable domain for analyzing and learning object interactions. We first present how behaviors are represented by finite state machines using the given input. Then, we analyze the impact of the knowledge about relations on the overall performance. Our analysis includes four different types of input: a knowledge base including part relations; spatial information; temporal information; and spatio-temporal information. We show that if a knowledge base about relations is provided, learning is accomplished to a desired extent. Our analysis also indicates that the spatio-temporal approach is superior to the spatial and the temporal approaches and gives close results to that of the knowledge-based approach.