On reversible Markov chains and maximization of directed information

In this paper, we consider a dynamical system, whose state is an input to a memoryless channel. The state of the dynamical system is affected by its past, an exogenous input, and causal feedback from the channel´s output. We consider maximizing the directed information between the input signal and the channel output, over all exogenous input distributions and/or dynamical system policies. We demonstrate that under certain conditions, reversibility of a Markov chain implies directed information is maximized. With this, we develop achievability theorems for channels with (infinite) memory as well as optimality conditions for sequential estimation of Markov processes through dynamical systems with causal feedback. We provide examples, which includes the exponential server timing channel and the trapdoor channel.