A multi-grid reinforcement learning method for energy conservation and comfort of HVAC in buildings

Online reinforcement learning often suffers from slow convergence and faults on early stages. In this paper, we propose a multi-grid method of Q-learning to handle these problems. It adopts a coarse model to fast converge to a good policy on early stages and then adopts a fine model to further improve the optimization result. This approach is applied to an optimal control problem of energy conservation and comfort of HVAC in buildings. The optimization algorithm is implemented in simulation using Matlab and EnergyPlus. The results show an improvement of our approach both in the convergence and the performance on early stages, comparing with the traditional Q-learning method for HVAC in the literature. By defining some modified Pareto domination indices, we demonstrate the superiority of multiple performances comprising the competing energy conservation and the comfort.