• DocumentCode
    1145773
  • Title

    Drift and monotonicity conditions for continuous-time controlled Markov chains with an average criterion

  • Author

    Guo, Xianping ; Hernández-Lerma, Onésimo

  • Author_Institution
    Dept. de Matematicas, CINVESTAV-IPN, Mexico City, Mexico
  • Volume
    48
  • Issue
    2
  • fYear
    2003
  • Firstpage
    236
  • Lastpage
    245
  • Abstract
    We give conditions for the existence of average optimal policies for continuous-time controlled Markov chains with a denumerable state-space and Borel action sets. The transition rates are allowed to be unbounded, and the reward/cost rates may have neither upper nor lower bounds. In the spirit of the "drift and monotonicity" conditions for continuous-time Markov processes, we propose a new set of conditions on the controlled process\´ primitive data under which the existence of optimal (deterministic) stationary policies in the class of randomized Markov policies is proved using the extended generator approach instead of Kolmogorov\´s forward equation used in the previous literature, and under which the convergence of a policy iteration method is also shown. Moreover, we use a controlled queueing system to show that all of our conditions are satisfied, whereas those in the previous literature fail to hold.
  • Keywords
    Markov processes; continuous time systems; convergence of numerical methods; decision theory; iterative methods; optimal control; probability; queueing theory; Borel action sets; average criterion; average optimal policies; continuous-time controlled Markov chains; controlled queueing system; denumerable state-space; deterministic policies; drift; extended generator approach; monotonicity conditions; optimal stationary policies; policy iteration method; primitive data; randomized Markov policies; reward/cost rates; Cities and towns; Control systems; Convergence; Cost function; Integral equations; Markov processes; Mathematics; Optimal control; Process control; Stochastic systems;
  • fLanguage
    English
  • Journal_Title
    Automatic Control, IEEE Transactions on
  • Publisher
    ieee
  • ISSN
    0018-9286
  • Type

    jour

  • DOI
    10.1109/TAC.2002.808469
  • Filename
    1178904