Function approximation for reinforcement learning based on reaction-diffusion equation on a graph

We propose applying the reaction-diffusion equation on a graph to the function approximation for reinforcement learning. which realizes adaptive resolution according to the complexity of the approximated value function. The function approximator expressed by nodes can change its resolution adaptively by distributing the nodes densely in the complex region of the state space with the proposed algorithm. A function is expressed in a plane. Each plane corresponds to a node. which is an element of the graph. Each node moves to diffuse the complexity of the approximated function in the neighborhood based on the reaction-diffusion equation. The complexity of the function is defined by the change of gradient. The simulation shows these two points: 1) The proposed algorithm provides the adaptability for function approximation. and 2) The function approximation improves the efficiency of the reinforcement learning.