Efficient estimation and control for Markov processes

Considers the problem of sequential control for a finite state and action Markovian decision process with incomplete information regarding the transition probabilities P∈P˜. Under suitable irreducibility assumptions for P˜, the authors construct adaptive policies that maximize the rate of convergence of realized rewards to that of the optimal (non adaptive) policy under complete information. These adaptive policies are specified via an easily computable index function, of states, controls and statistics, so that one takes a control with the largest index value in the current state in every period