Some results on ergodic and adaptive control of hidden Markov models

The dynamics of a discrete time, state process are assumed to depend on the current value of the state of a possibly unobserved hidden Markov model. Both the state and the hidden process are controlled with a control that depends on the available observations. An ergodic or average cost per unit time control problem is solved making some regularity assumptions. If the transition operators of the state and the hidden Markov processes depend on an unknown random variable with a known probability law, then the Bayesian adaptive control approach is used to construct an almost optimal adaptive control.