Near-optimality bounds for greedy periodic policies with application to grid-level storage

This paper is concerned with periodic Markov Decision Processes, as a simplified but already rich model for nonstationary infinite-horizon problems involving seasonal effects. Considering the class of policies greedy for periodic approximate value functions, we establish improved near-optimality bounds for such policies, and derive a corresponding value-iteration algorithm suitable for periodic problems. The effectiveness of a parallel implementation of the algorithm is demonstrated on a grid-level storage control problem that involves stochastic electricity prices following a daily cycle.