Title :
Robust Optimality for Discounted Infinite-Horizon Markov Decision Processes With Uncertain Transition Matrices
Author :
Li, Baohua ; Si, Jennie
Author_Institution :
Dept. of Electr. Eng., Arizona State Univ., Tempe, AZ
Abstract :
We study finite-state, finite-action, discounted infinite-horizon Markov decision processes with uncertain transition matrices in the deterministic policy space. The transition matrices are classified as either independent or correlated. A generalized robust optimality criterion which can be degenerated to some popular optimality criteria is proposed, under which an optimal or near-optimal policy exists for any uncertain transition matrix. Theorems are developed to guarantee a stationary policy being optimal or near-optimal in the deterministic policy space.
Keywords :
Markov processes; decision theory; infinite horizon; matrix algebra; uncertain systems; deterministic policy space; discounted infinite-horizon Markov decision process; finite-state finite-action MDP; generalized robust optimality criterion; uncertain transition matrix; Cost function; Estimation error; Inventory control; Quality control; Robust control; Robustness; Space stations; Markov decision processes; robust optimality criterion; uncertain transition matrix;
Journal_Title :
Automatic Control, IEEE Transactions on
DOI :
10.1109/TAC.2008.930182
