Q-learning for optimal deployment strategies of frequency controllers using the aggregated storage of PHEV fleets

As more renewable energy sources (RES) get connected to the electric power network, the stability of the system is gradually put into increasing risk. RES lack stabilizing characteristics, such as inertia, which are inherent to conventional synchronous machines. Mimicking inertia techniques, by appropriately controlling an external power source such as a large battery storage, improve the stability of the system. Since large battery storage is costly, a distributed battery storage, based on Plug-In Hybrid Electric Vehicles (PHEVs) seems an attractive option. This paper investigates the use of the aggregated storage from large, distributed PHEV fleets for frequency control by inertia-mimicking techniques. The focus is on the saturation limits of the aggregated storage, as well as on the controller placement and speed. An algorithm based on Q-learning is developed to determine an optimal controller placement strategy.