The entropy of Markov trajectories

The idea of thermodynamic depth put forth by S. Lloyd and H. Pagels (1988) requires the computation of the entropy of Markov trajectories. Toward this end, the authors consider an irreducible finite state Markov chain with transition matrix P and associated entropy rate H(X)=-Σ_i,j μ_iP_ij log P_ij where μ is the stationary distribution given by the solution of μ=μP. A trajectory T_ij of the Markov chain is a path with initial state i, final state j, and no intervening states equal to j. It is shown that the entropy H(T_ii) of the random trajectory originating and terminating in state i is given by H( T_ii)=H(X)/μ_i. Thus the entropy of the random trajectory T_ii is the product of the expected number of steps 1/μ_i to return to state i and the entropy rate H(X) per step for the stationary Markov chain. A general closed form solution for the entropies H(T_ij) is given by H=K-K˜+H_Δ, where H is the matrix of trajectory entropies H_ij =H(T_ij); K=(I-P +A)^-1 (H*-H_Δ); K˜ is a matrix in which the ijth element K˜_ij equals the diagonal element K_jj of K; A is the matrix of stationary probabilities with entries A _ij=μ_j; H* is the matrix of single-step entropies with entries H*_ij=H(P _i)=-Σ_k P_ik log P_ik; and H_Δ is a diagonal matrix with entries (H_Δ)_ii=H (X)/μ_i