Online learning of foot placement

Summary form only given. Using machine learning algorithms to improve imperfect modeling and to estimate parameters with the hardware in the loop is a promising way to achieve balanced and dynamic gaits. Similar to the pendulum-cart model, a biped can accelerate its center of mass and control its angular momentum by modifying its step size to maintain balance. Using this simple concept, we derive a gradient function and use it to update the generated step size depending on the trunk angle we measure at the end of a step. Step sizes are represented by a function approximator that is updated online during walking. In this manner, we obtain a fast and robust online-learning technique that enables a simulated biped to learn how to maintain balance in the presence of strong disturbances, and to follow a reference footstep plan. The video shows an experiment where we disturb the robot with push impulses from the back. The algorithm learns how to absorb the pushes and to return to a stationary walk with only a few experiences. Initializing the step controller with an analytically engineered controller [1, 2] and learning only an offset to its step size output improves the learning performance.