Multi-objective fitted Q-iteration: Pareto frontier approximation in one single run

We present a novel batch-mode Reinforcement Learning approach for the design of optimal controllers in the presence of multiple objectives. The algorithm is an extension of Fitted Q-iteration (FQI) that enables to design the controller for all the linear combinations of preferences (weights) assigned to the objectives in a single run. The key idea of multi-objective FQI (MOFQI) is to enlarge the continuous approximation of the value function, which is performed by single-objective FQI over the state-control space, also to the weight space. The bacth-mode nature of the algorithm makes it possible the enrichment of the learning data with nearly no additional computational cost with respect to a single-objective formulation on the same system. The approach was tested on a simple test case study concerning the optimal operation of a two-objective water reservoir, where MOFQI algorithm proved to be computationally preferable over repeatedly running FQI for different weight values when more than five points on the Pareto frontier are considered.