Game-theoretic-reinforcement-adaptive neural controller for nonlinear systems

This paper is concerned with application of a game-theoretic formulation for direct adaptive optimal control of nonlinear systems affected by bounded noise/disturbances. Almost all of the reinforcement learning (RL) approaches employ a Markov decision process (MDP) setting, which is inadequate for handling noise/disturbances as it disallows an explicit representation of noise/disturbances. The work in this paper addresses this problem by showing how a game theoretic RL formulation can effectively cope with disturbances with no preliminary offline learning. The work demonstrates the application of neural networks (NN) in a Markov game setup. We provide empirical results of applying the approach on an inverted pendulum swing-up task and compare its performance against Q learning. The results bring out the superiority of the Markov game formulation for direct adaptive optimal control over the MDP setting. Further, the results demonstrate that NN could be used as an effective tool for speeding up learning in large Markov games and for extending the game theoretic direct adaptive optimal control to a continuous state space