On partially observed stochastic shortest path problems

Author

Patek, Stephen D.

Author_Institution

Dept. of Syst. & Inf. Eng., Univ. of Virginia, VA, USA

Volume

5

fYear

2001

fDate

2001

Firstpage

5050

Abstract

We analyze a class of partially observed stochastic shortest path problems. These are terminating Markov decision process with imperfect state information that evolve on an infinite time horizon and have a total cost criterion. For wellposedness, we make reasonable stochastic shortest path type assumptions: (1) the existence of a policy that guarantees termination with probability one; and (2) the property that any policy that fails to guarantee termination has infinite expected cost from some initial state. We also assume that termination is perfectly recognized. We establish the existence of a stationary optimal policy along with the existence of a unique bounded solution to Bellman´s equation. We also reveal the convergence properties of value and policy iteration. For the case where policies exist that do not guarantee termination, the dynamic programming operator fails to be a contraction mapping with respect to any norm, somewhat complicating the analysis

Keywords

Markov processes; convergence of numerical methods; dynamic programming; probability; search problems; Markov decision processes; convergence; dynamic programming; optimal search; probability; stochastic shortest path problems; Convergence; Costs; Dynamic programming; Equations; Failure analysis; Information analysis; Shortest path problem; Stochastic processes; Stochastic systems; Systems engineering and theory;

fLanguage

English

Publisher

ieee

Conference_Titel

Decision and Control, 2001. Proceedings of the 40th IEEE Conference on

Conference_Location

Orlando, FL

Print_ISBN

0-7803-7061-9

Type

conf

DOI

10.1109/.2001.981011

Filename

981011