Multi-agent hierarchical architecture modeling kinematic chains employing continuous RL learning with fuzzified state space

In the context of multi-agent systems, we are proposing a hierarchical robot control architecture that comprises artificial intelligence (AI) techniques and traditional control methodologies, based on the realization of a learning team of agents in a continuous problem setting. In a multi-agent system, action selection is important for cooperation and coordination among the agents. By employing reinforcement learning (RL) methods in a fuzzified state-space, we accomplish to design a control architecture and a corresponding methodology, engaged in a continuous space, which enables the agents to learn, over a period of time, to perform sequences of continuous actions in a cooperative manner, in order to reach their goal without any prior generated task model. By organizing the agents in a nested architecture, as proposed in this work, a type of problem-specific recursive knowledge acquisition is attempted. Furthermore, the agents try to exploit the knowledge gathered in order to be in position to execute tasks that indicate certain degree of similarity. The agents correspond in fact to independent degrees of freedom of the system, and achieve to gain experience over the task that they collaboratively perform, by exploring and exploiting their state-to-action mapping space. A numerical experiment is presented in this paper, performed on a simulated planar 4 degrees of freedom (DOF) manipulator, in order to evaluate both the proposed hierarchical multi-agent architecture as well as the proposed methodological framework. It is anticipated that such an approach can be highly scalable for the control of robotic systems that are kinematically more complex, comprising multiple DOFs and potentially redundancies in open or closed kinematic chains, particularly dexterous manipulators.