Intelligent Robot Moving Control Using Neural Networks and Biological Motor Adaptive Model

This article describes a novel approach for adaptive optimal control and demonstrates its application to a variety of systems, including motion control learning for legged robots. The new controller, called FOX (Fairly obvious extension), uses a modified form of Albus’s CMAC (Cerebellar model articulation controller) neural network. It is trained to generate control signals that minimize a system’s performance error. A theoretical consideration of the adaptive control problem is used to show that FOX must assign each CMAC weight an eligibility value which controls how that weight is updated. FOX thus implements a kind of reinforcement learning which makes it functionally similar to the cerebellum (a part of the brain that modulates movement). A highly efficient implementation is described which makes FOX suitable for on-line control. FOX requires a small amount of dynamical information about the system being controlled: the system’s impulse response is used to choose the rules that update the eligibility values. A FOX-based controller design methodology is developed, and FOX is tested on four control problems: controlling a simulated linear system, controlling a model gantry crane, balancing an inverted pendulum on a cart, and making a wheeled robot follow a path. In each case FOX is effective: it associates sensor values with (and anticipates) the correct control actions, it compensates for system nonlinearities, and it provides robust control as long as the training is comprehensive enough. FOX is also applied to the control of a simulated hopping monoped, and a walking biped. FOX learns parameters that fine tune the movements of pre-programmed controllers, in a manner analogous to the cerebellar modulation of spinal cord reflexes in human movement. The robots are successfully taught how to move with a steady gait along flat ground, in any direction, and how to climb and descend slopes