Optimal and Near-Optimal Incentive Strategies in the Hierarchical Control of Markov Chains

This paper considers Markovian Stackelberg problems with one leader and N followers. Firstly, an algorithm is proposed to compute optimal affine incentive strategy for the leader and Nash reactions of the followers, for general finite state Markov cahins, under the average-cost-per-stage criteria. Next, this algorithm is analyzed in the context of weakly-coupled Markov chains to compute near-optimal strategies from a reduced-order aggregate model. The robustness of the near-optimal solution is established, and the multimodel feature of the computational algorithm is highlighted.