Learning based bidding strategy for HVAC systems in double auction retail energy markets

In this paper, we propose a reinforcement learning bidding strategy for controlling and coordinating HVAC systems in a double auction market. The bidding strategy does not require a specific model-based representation of behavior, i.e., a functional form to translate indoor house temperatures into bid prices. The results from reinforcement learning based approach are compared with the HVAC bidding approach used in the AEP gridSMART^® smart grid demonstration project and it is shown that the model-free (learning based) approach tracks well the results from the model-based behavior. Successful use of model-free approaches to represent device-level economic behavior may advance similar approaches to represent behavior of more complex devices or groups of diverse devices, such as the aggregate behavior of responding devices in a building.