اﻓﺰاﯾﺶ ﺳﻮدآوري ﺑﺎزار ﺷﺒﮑﻪ ﻫﺎي ﻫﻮﺷﻤﻨﺪ ﺑﺮق ﺑﺎ ﺗﮑﻨﯿﮏ ﯾﺎدﮔﯿﺮي ﺗﻘﻮﯾﺘﯽ ﻋﻤﻠﮕﺮ-ﻧﻘﺎد

Profit increasing in smart grid market via actor-critic reinforcement learning

ﭼﮑﯿﺪه: ﺑﺎزار ﺷﺒﮑﻪ ﻫﺎي ﻫﻮﺷﻤﻨﺪ ﺑﺮق ﭘﯿﭽﯿﺪه و ﭘﻮﯾﺎﺳﺖ. ﮐـﺎرﮔﺰاران ﮐـﻪ واﺳـﻄﻪ ﮔـﺮان ﻓـﺮوش ﺑـﺮق ﺑـﯿﻦ ﺧـﺮده ﻓﺮوﺷـﯽ ﻫـﺎ و ﻋﻤﺪه ﻓﺮوﺷﯽ ﻫﺎ ﻫﺴﺘﻨﺪ ﺑﻪ ﺻﻮرت ﮔﺴﺘﺮده اي در ﺑﺎزارﻫﺎي ﺟﺪﯾﺪ ﺷﺒﮑﻪ ﻫﺎي ﻫﻮﺷﻤﻨﺪ ﺑﻪ ﮐﺎر ﮔﺮﻓﺘﻪ ﻣـﯽ ﺷـﻮﻧﺪ. ﺑـﻪ ﻋﻠـﺖ ﭘﯿﭽﯿـﺪﮔﯽ و ﺗﻮزﯾﻊ ﺷﺪﮔﯽ ذاﺗﯽ ﺑﺎزار در ﺷﺒﮑﻪ ﻫﺎي ﻫﻮﺷﻤﻨﺪ روﯾﮑﺮدﻫﺎي اﺳﺘﻔﺎده از ﺳﯿﺴﺘﻢ ﻫﺎي ﭼﻨﺪﻋﺎﻣﻠﻪ ﺑﺮاي ﺣﻞ ﻣﺴﺎﺋﻞ آن ﻣﻨﺎﺳﺐ اﺳـﺖ. در اﯾﻦ روﯾﮑﺮدﻫﺎ ﻣﯽ ﺗﻮاﻧﯿﻢ ﻋﺎﻣﻞ ﻫﺎي ﺧﻮدﻣﺨﺘﺎري داﺷﺘﻪ ﺑﺎﺷﯿﻢ ﮐﻪ ﺑﻪ ﺻﻮرت 24 ﺳﺎﻋﺘﻪ درﺣﺎل ﺗﺒﺎدل اﻃﻼﻋﺎت ﺑﺎ دﯾﮕﺮ ﻋﺎﻣﻞ ﻫﺎ ﻫﺴﺘﻨﺪ. اﯾﻦ ﻋﺎﻣﻞ ﻫﺎ ﺑﺎ ﭼﺎﻟﺶ ﻫﺎي اﺳﺎﺳﯽ ﺷﺎﻣﻞ اﻟﮕﻮي ﻣﺼﺮف ﻣﺘﻨﻮع ﻣﺸﺘﺮﯾﺎن، ﺗﻐﯿﯿﺮ ﻗﯿﻤﺖ ﺑﺎ ﺗﻮﺟﻪ ﺑﻪ اﻟﮕـﻮي ﻣﺼـﺮف ﻣﺸـﺘﺮﯾﺎن و ﻣﯿـﺰان ﻣﺼﺮف ﺑﺮق در ﻃﻮل ﺷﺒﺎﻧﻪ روز ﻣﻮاﺟﻪ اﻧﺪ. ﻫﺪف ﻣﺎ در اﯾﻦ ﻣﻘﺎﻟﻪ اﯾﻦ اﺳﺖ ﮐﻪ ﺿﻤﻦ ﻣﺪل ﮐﺮدن اﺟـﺰاي ﺑـﺎزار ﺑـﺮق ﺑـﺎ ﺳﯿﺴـﺘﻢ ﻫـﺎي ﭼﻨﺪﻋﺎﻣﻠﻪ، ﺑﺎ اراﺋﻪ روﺷﯽ ﻣﺒﺘﻨﯽ ﺑﺮ ﯾﺎدﮔﯿﺮي ﻋﺎﻣﻞ ﻫﺎ ﺳﻮدآوري در ﺑﺎزار ﺷﺒﮑﻪ ﻫﺎي ﺑﺮق را اﻓﺰاﯾﺶ دﻫﯿﻢ. در روش ﭘﯿﺸـﻨﻬﺎدي اﺑﺘـﺪا ﻣﺴﺎﻟﻪ ﺗﻨﻮع ﻣﺼﺮف ﻣﺸﺘﺮﯾﺎن را ﺑﺎ اﻧﺠﺎم ﯾﮏ روش ﺧﻮﺷﻪ ﺑﻨﺪي ﻣﺘﻮاﻟﯽ ﻣﻨﺎﺳﺐ دادهﻫﺎي ﺳﺮي زﻣﺎﻧﯽ ﭘﺮدازش ﻣﯽ ﮐﻨﯿﻢ. ﺳﭙﺲ ﺑﺮاي ﻫﺮ ﮔﺮوه ﺧﻮﺷﻪ ﺑﻨﺪي ﺷﺪه ﺑﻪ ﺻﻮرت ﻣﺠﺰا ﯾﮏ روش ﯾﺎدﮔﯿﺮي ﺗﻘﻮﯾﺘﯽ ﺳﯿﺎﺳﺖ ﻓﻌﺎل ﺑﺎ ﻋﻨﻮان ﯾﺎدﮔﯿﺮي ﺗﻘﻮﯾﺘﯽ ﻋﻤﻠﮕـﺮ - ﻧﻘـﺎد ﺑـﻪ ﮐـﺎر ﻣﯽ ﺑﺮﯾﻢ. درﻧﻬﺎﯾﺖ ﺗﺎﺛﯿﺮ ﺗﻐﯿﯿﺮ ﭘﺎداش را در ﺳﻮد ﺣﺎﺻﻠﻪ ارزﯾﺎﺑﯽ ﻣﯽ ﮐﻨﯿﻢ و ﺑﺮاي ﻫﺮ ﺧﻮﺷﻪ ﺗﻌﺮﻓﻪ اي ﻣﻄﺎﺑﻖ ﺑﺎ زﻣﺎن ﻣﺼﺮف ﻣﺮﺑﻮﻃﻪ ﺑـﻪ ﺻﻮرت ﺳﺎﻋﺘﯽ اراﺋﻪ ﻣﯽ دﻫﯿﻢ.

The electricity smart grid market is complex and dynamic. Brokers, which mediate the sale of electrical power between retailers and wholesalers, are widely used in new markets for smart grids. Due to the complexity and distribution properties of the market in smart grid networks, multi-agent systems are appropriate to solve its problems. In these approaches, we have autonomous agents exchanging information with other agents all 24 hours of a day. These agents encounter major challenges including diverse consumption patterns of consumers, price changing according to consumption patterns, and the amount of electricity consumed during the day. In this paper our goal is to increase profit in the electricity grid market while modeling the components of the electricity market with multi-agent systems. In the proposed method, we first process the customer diversity using a sequential clustering method suitable for time series data. Then, for each cluster, we apply an active policy reinforcement learning algorithm named Actor-Critic reinforcement learning. Finally, we evaluate the impact of the reward shaping on the profit earnings and we offer an hourly tariff for each cluster according to their respective consumption time