Multiple time scale decomposition and state space aggregation of controlled Markov processes

For large scale systems multistage optimization over a long horizon is most conveniently done in a hierarchical fashion: first a long range time-space aggregated problem is solved and then a short range problem is solved. In some cases, a medium range optimization problem is also defined. Operation scheduling for nuclear-hydro-thermal power systems is a typical example. The above represents an intuitive description of a possible hierarchical decomposition of the operation scheduling problem. In this paper we present a mathematical treatment in terms of a controlled finite state Markov process. Our treatment indicates how the approximate decomposition of the time scale and the aggregation of the state space follows from properties of the probability transition matrix of the Markov process.