Model-based learning with Bayesian and MAXQ value function decomposition for hierarchical task

How to improve efficiency of learning is always the key issue for implementation of reinforcement learning. This paper makes use of advantages of both hierarchical learning and model-based learning, so that an improved algorithm, named Bayesian-MAXQ learning, is introduced, in which several modifications are developed to solve the value update of hierarchy, while the possible performance damages brought by prioritized sweeping is reduced to trivial. The simulation results show that, Bayesian-MAXQ learning performs with high efficiency, and it can serve as a good framework for further study on hierarchical model-based learning.