Average Optimal Stationary Policies and Linear Programming in Countable Space Markov Decision Processes

We relate average optimal stationary policies in countable space Markov decision processes and optimal solutions of an associated infinite dimensional linear program. Using the theory of linear programming in abstract spaces, sufficient conditions for existence of optimal solutions are presented and some previous ones are interpreted.