Research on Dynamic Bayesian Network in the Nonhomogenous Markov Decision Processes

A new method of modeling the nonhomogenous Markov decision processes with dynamic Bayesian networks (DBNs) is proposed so that DBNs can be applied in more wide fields. In this paper, the extended hidden variables are introduced into the evolutional process so as to build Markov models, a structure learning algorithm of DBNs is provided in the presence of the incomplete data and the extended hidden variables, the sufficient statistics of posterior time slices are estimated using Bayesian probability statistical method, and then the time-variant transition probabilities are learned with both current sufficient statistics and estimated sufficient statistics. The theoretical analysis and simulation results validate the correctness of the proposed method