Denumerable controlled Markov chains with strong average optimality criterion: bounded and unbounded costs

Studies discrete-time controlled stochastic systems, modeled by controlled Markov chains (CMC) with denumerable state space and compact actions space, and with an infinite planning horizon. One optimality criterion often used is the long-run expected average cost (AC). The authors´ main interest in this paper is to study the relation between the notions of AC optimality and strong average cost (SAC) optimality. The latter criterion is introduced to assess the performance of a policy over long but finite horizons, as well as in the long-run average sense. The authors show that for bounded one-stage cost functions, Sennott´s conditions are sufficient to guarantee that every AC optimal policy is also SAC optimal. On the other hand, a detailed counterexample is given that shows that the latter result does not extend to the case of unbounded cost functions