Optimal adaptive controllers for unknown systems

We consider the problem of adaptively controlling an unknown Markov Chain. We provide a heuristic argument of why a certain family of adaptive controllers is well-behaved. Specifically, an intuitive explanation is offered (which supplements an earlier mathematical treatment) to show that the adaptive control law converges in a Cesaro sense to the set of optimal control laws and also that the long-term average cost incurred by the adaptive controllers is optimal.