Adaptive control of a Markov chain over a finite parameter set without continuity assumptions on the control laws

In this note we provide an extension of some results given in [1] to the case without continuity assumptions on control laws w.r.t, state variable. This extension is important for applications since it is rarely the case that optimal stationary control laws are smooth enough (e.g., continuous), as a function of state, to satisfy continuity assumptions 1i/, 1iii/, and 1v of [1].