Title :
On partially observed stochastic shortest path problems
Author :
Patek, Stephen D.
Author_Institution :
Dept. of Syst. & Inf. Eng., Univ. of Virginia, VA, USA
Abstract :
We analyze a class of partially observed stochastic shortest path problems. These are terminating Markov decision process with imperfect state information that evolve on an infinite time horizon and have a total cost criterion. For wellposedness, we make reasonable stochastic shortest path type assumptions: (1) the existence of a policy that guarantees termination with probability one; and (2) the property that any policy that fails to guarantee termination has infinite expected cost from some initial state. We also assume that termination is perfectly recognized. We establish the existence of a stationary optimal policy along with the existence of a unique bounded solution to Bellman´s equation. We also reveal the convergence properties of value and policy iteration. For the case where policies exist that do not guarantee termination, the dynamic programming operator fails to be a contraction mapping with respect to any norm, somewhat complicating the analysis
Keywords :
Markov processes; convergence of numerical methods; dynamic programming; probability; search problems; Markov decision processes; convergence; dynamic programming; optimal search; probability; stochastic shortest path problems; Convergence; Costs; Dynamic programming; Equations; Failure analysis; Information analysis; Shortest path problem; Stochastic processes; Stochastic systems; Systems engineering and theory;
Conference_Titel :
Decision and Control, 2001. Proceedings of the 40th IEEE Conference on
Conference_Location :
Orlando, FL
Print_ISBN :
0-7803-7061-9
DOI :
10.1109/.2001.981011
