Optimal strategies for adaptive zero-sum average Markov games

We consider a class of discrete-time two person zero-sum Markov games with Borel state and action spaces, and possibly unbounded payoffs. The game evolves according to the recursive equation x n + 1 = F ( x n , a n , b n , ξ n ) , n = 0 , 1 , … , where the disturbance process { ξ n } is formed by independent and identically distributed R k -valued random vectors, which are observable but their common density ρ ∗ is unknown for both players. Combining suitable methods of statistical estimation of ρ ∗ with optimization procedures, we construct a pair of average optimal strategies.