Decentralized Dynamic Workflow Scheduling for Grid Computing using Reinforcement Learning

The workflow enactment engine is used to execute grid workflow on heterogeneous and distributed resources. However, in the literature, the efficient workflow scheduling algorithm designating resources to tasks in a dynamic environment has not been carefully investigated. In the paper, a decentralized dynamic workflow scheduling algorithm using reinforcement learning (DDWS-RL) is proposed. The on-line model-free RL algorithm is embedded into the decentralized just in-time scheduling system together with a RL agent. At the time of tasks execution, the decentralized task schedulers query information from the RL agent, designate resources to tasks and update the RL agent. To evaluate the efficiency of the DDWS-RL algorithm, a real grid network is built. The Globus Toolkit 2.4 is installed as the middleware for the testbed and the workflow enactment engine and the DDWS-RL algorithm are realized in Java programming. The experiment results show that the proposed DDWS-RL algorithm converges to the theoretical shortest execution time of the workflow in the homogeneous environment. In the heterogeneous environment, the algorithm reaches the sub-optimal execution time due to the self-interest of the independent learner applied in task scheduler