A New Optimization Model for the Construction of Markov Chains

We study the problem of construction of the transition probability matrix of a Markov chain from a given steady-state probability distribution. We note that for this inverse problem, there are many possible transition probability matrices sharing the same steady-state probability distribution. Therefore extra constraint has to be introduced so as to narrow down the set of solutions or even a unique solution. We propose to consider maximizing the generalized entropy rate of the Markov chain with a penalty cost as the extra criterion. We first give a mathematical formulation of the inverse problem as a maximization problem. We then apply the Lagrange multiplier method to the original problem. Numerical examples in contrast with are given to demonstrate the effectiveness of the proposed method.