DocumentCode :
57451
Title :
An Argument for the Bayesian Control of Partially Observable Markov Decision Processes
Author :
Vargo, Erik ; Cogill, Randy
Author_Institution :
Dept. of Syst. & Inf. Eng., Univ. of Virginia, Charlottesville, VA, USA
Volume :
59
Issue :
10
fYear :
2014
fDate :
Oct. 2014
Firstpage :
2796
Lastpage :
2800
Abstract :
This technical note concerns the control of partially observable Markov decision processes characterized by a prior distribution over the underlying hidden Markov model parameters. In such instances, the control problem is commonly simplified by first choosing a point estimate from the model prior, and then selecting the control policy that is optimal with respect to the point estimate. Our contribution is to demonstrate, through a tractable yet nontrivial example, that even the best control policies constructed in this manner can significantly underperform the Bayes optimal policy. While this is an operative assumption in the Bayes-adaptive Markov decision process literature, to our knowledge no such illustrative example has been formally proposed.
Keywords :
Bayes methods; adaptive control; decision theory; hidden Markov models; optimal control; stochastic systems; Bayes-adaptive Markov decision process; Bayesian control; adaptive control; hidden Markov model parameters; optimal Bayes optimal policy; optimal control policy; partially observable Markov decision processes; point estimate; stochastic optimal control; Adaptation models; Bayes methods; Computational modeling; Hidden Markov models; Markov processes; Standards; Uncertainty; Adaptive control; Markov processes; stochastic optimal control; uncertain systems;
fLanguage :
English
Journal_Title :
Automatic Control, IEEE Transactions on
Publisher :
ieee
ISSN :
0018-9286
Type :
jour
DOI :
10.1109/TAC.2014.2314527
Filename :
6781561
Link To Document :
بازگشت