Probabilistic roadmap with self-learning for path planning of a mobile robot in a dynamic and unstructured environment

This paper presents a new path planning method for a mobile robot in an unstructured and dynamic environment. The method consists of two steps: first, a probabilistic roadmap (PRM) is constructed and stored as a graph whose nodes correspond to a collision-free world state for the robot; second, Q-learning-a method of reinforcement learning, is integrated with PRM to determine a proper path to reach the goal. In this manner, the robot is able to use past experience to improve its performance in avoiding not only static obstacles but also moving obstacles, without knowing the nature of the movements of the obstacles. The developed approach is applied to a simulated robot system. The results show that the hybrid PRM-Q path planner is able to converge to the right path successfully and rapidly.