Determining the Optimal Strategies for Stochastic Positional Games with Discounted Payoffs

We formulate and study a class of stochastic positional games applying the concept of positional games to finite state space Markov decision processes with expected total discounted cost criterion. Necessary and sufficient conditions for the existence of Nash equilibria in stochastic positional games with discounted payoffs are proven and some approaches for determining the optimal stationary strategies of the players are analyzed. Efficient iterative algorithms for determining the optimal strategies of the players for the considered class of games are proposed and grounded. These results extend Nash equilibria conditions for deterministic positional games and can be used for studying Shapley stochastic games with discounted payoffs.