Using Q-learning to model bidding behaviour in electricity market simulation

While the choice between two pricing rules, namely Uniform pricing rule and Pay-as-bid pricing rule, has led to a continuous debate in the electricity market establishment process, little attention has been paid to the Vickrey pricing rule. This paper presents an agent-based model to examine the employment of Uniform and Vickrey pricing rules in a deregulated electricity market. Using Q-learning in repetitive trading process, generator agents learn the market characteristics and seek to maximise their revenue by exploring bidding strategies. A look up table is utilised to memorise agents´ bidding experience that help the agents improve their strategies. Supply quantity withholding and generators´ collusion phenomenon have been observed in this study under certain market arrangements. The implication of these two pricing rules on the total dispatch costs and generators´ profit are discussed in this paper.