Control of Dynamic Systems under the Influence of Singularly Perturbed Markov Chains

This work is concerned with nearly optimal controls of nonlinear dynamic systems under the influence of singularly perturbed Markov chains. The underlying Markov chains have fast and slow components and their states can be divided into a number of groups. Within each group of states, the chain varies in a fast pace whereas the jumps from one group to another occur relative infrequently. To obtain the desired optimality, the states of the chain are naturally aggregated in accordance with the transition rates; i.e., replacing the states in a group by a single state to obtain an average system. Then the averaged system is used as a reference to develop the nearly optimal control for the actual system via comparison control methods. The technique used is the method of weak convergence together with the utilization of relaxed control representation.