Fuzzy Q-learning: a new approach for fuzzy dynamic programming

Fuzzy reinforcement learning (FRL) involves “jump starting” reinforcement learning with fuzzy logic rules. By using FRL, prior domain knowledge, which may be very approximate and imprecise, can be expressed in terms of fuzzy rules and refined later through the learning process. In this paper, we develop a new algorithm called fuzzy Q-learning (or FQ-Learning) which extends Watkin´s Q-learning method. It can be used for decision processes in which the goals and/or the constraints, but not necessarily the system under control, are fuzzy in nature. An example of a fuzzy constraint is: “the weight of object A must not be substantially heavier than w” where w is a specified weight. Similarly, an example of a fuzzy goal is: “the robot must be in the vicinity of door k”. We show that FQ-learning provides an alternative solution to this problem which is simpler than the Bellman-Zadeh´s fuzzy dynamic programming approach. We apply the algorithm to a multistage decision making problem and a navigation task