Acquiring a broad range of empirical knowledge in real time by temporal-difference learning

Several robot capabilities rely on predictions about the temporally extended consequences of a robot´s behaviour. We describe how a robot can both learn and make many such predictions in real time using a standard algorithm. Our experiments show that a mobile robot can learn and make thousands of accurate predictions at 10 Hz. The predictions are about the future of all of the robot´s sensors and many internal state variables at multiple time-scales. All the predictions share a single set of features and learning parameters. We demonstrate the generality of this method with an application to a different platform, a robot arm operating at 50 Hz. Here, learned predictions can be used to measurably improve the user interface. The temporally extended predictions learned in real time by this method constitute a basic form of knowledge about the dynamics of the robot´s interaction with the environment. We also show how this method can be extended to express more general forms of knowledge.