Effect of force load in hand reaching movement acquired by reinforcement learning

It has been known that when a human moves its hand to a target, the trajectory becomes almost a straight line from the start point to the target. When a viscosity force field is loaded to the hand unexpectedly, it is pulled toward the force direction once and then goes back to the target. However, after the learning in the force field, the trajectory becomes a straight line again, and when the force field is removed, it is pulled toward the opposite direction of the force that was loaded to the hand. This is called after-effect. In this paper, a neural network, whose inputs are visual sensory signals and state of manipulator, and whose outputs are joint torques, was trained by reinforcement learning. The effect of the first force field exposure and after-effect could be observed. This means that the system obtains inverse dynamics of its hand and environment in the neural network through reinforcement learning. Further, when the neural network learned with a random force at every trial, it became to control its hand based on feedback control rather than feedforward control.