DocumentCode :
3476740
Title :
On partially observable Markov decision processes with an average cost criterion
Author :
Fernández-Gaucherand, Emmanuel ; Arapostathis, Aristotle ; Marcus, Steven I.
Author_Institution :
Dept. of Electr. & Comput. Eng., Texas Univ., Austin, TX, USA
fYear :
1989
fDate :
13-15 Dec 1989
Firstpage :
1267
Abstract :
Consideration is given to partially observable Markov decision processes with finite or countable (core) state and observation spaces and finite control space. Following a standard approach, an equivalent completely observed problem is formulated, with the same finite control space but with an uncountable state space, namely, the space of probability distributions of the original core state space. It is observed that some characteristics induced in the original problem due to the finiteness, or countability, of the spaces involved are retained by the equivalent problem. Sufficient conditions are derived for a bounded solution to the average-cost optimality equation to exist. These results are illustrated in the context of machine replacement problems. Structural properties for average-cost policies are obtained for a two-state replacement problem and are similar to available results for discount optimal policies. The set of assumptions used seems to be significantly less restrictive than others currently available
Keywords :
Markov processes; decision theory; optimisation; probability; state-space methods; Markov decision processes; average cost criterion; countability; finite control space; finiteness; machine replacement problems; observation spaces; optimality; probability distributions; state space; Application software; Communication system control; Cost function; Equations; Optimal control; Probability distribution; Random processes; State-space methods; Stochastic processes; Topology;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Decision and Control, 1989., Proceedings of the 28th IEEE Conference on
Conference_Location :
Tampa, FL
Type :
conf
DOI :
10.1109/CDC.1989.70341
Filename :
70341
Link To Document :
بازگشت