Macro-Actions in Model-Free Intelligent Control with Application to pH Control

MFIC (Model-Free Intelligent Control) is a technique, based on Reinforcement Learning, previously proposed by the authors to control processes without needing a precalculated model. In standard reinforcement learning algorithms (including MFIC), the interaction between an agent and the environment is based on a fixed time scale: during learning, the agent can select several primitive actions depending on the system state. This creates the problem of selecting a suitable fixed time scale to select control actions, to trade off accuracy in control against learning complexity and flexibility. A novel solution to this problem is presented in this paper: Macro-actions, that incorporate a general closed-loop policy and temporal extended actions. The application of macro actions on a laboratory plant of pH process shows that the proposed MFIC learns to control adequately the neutralization process, with reduced computational effort.