A Direct Utility Adaptive Critic (DUAC) algorithm for power plant load management

This paper presents a Direct Utility Adaptive Critic (DUAC) algorithm applied to a power plant load management problem. The DUAC algorithm is an enhancement of the original Heuristic Dynamic Programming (HDP) Adaptive Critic Design (ACD) algorithm into a simpler and more robust controller. Typical ACD algorithms model dynamic systems with time-delayed states and action inputs and due to this the Action Network training procedure is a complex BackPropagation-Through-Time (BPTT) process. Also required in typical ACD algorithms is a dedicated Critic Network training process for different control sequences before the Action Network training procedure. The DUAC algorithm, presented in this paper, simplifies the Adaptive Critic algorithm by 1) eliminating the complex BPTT process for training the Action Network and 2) replacing the Critic Network with the user-defined utility function directly. Due to these changes the utility-action gradient typically required to train the Action Network is based on direct result of two utility values with respect to two action inputs. The replacement of the Critic Network with the user-defined utility function ensures better control accuracy since Critic Network modeling provides only approximations of the utility function. The DUAC algorithm was tested for time-varying consumer loads on an RMS voltage analogous s-domain model of the power plant created in Simulink using the SimPowerSystems toolbox. Test results indicated that the DUAC algorithm was able to derive an Action Network that controlled the power plant model to an output RMS voltage fluctuation variance of the order of no more than 10^-3. This result can prove to be an essential step in load balancing problems in Smart Grids.